Tuesday, May 28, 2013

Pushing and pulling

I'm mystified. The following definition of a program is efficient to execute:

sealed trait Program[A]
case class Result[A](x: A) extends Program[A]
case class NeedsInput[A](f: String => Program[A]) extends Program[A]

and in fact has a not bad definition of order():

def order[A1,A2](p1: Program[A1], p2: Program[A2]):
        Program[Either[(A1,Program[A2]),(A2,Program[A1])]] =
    p1 match {
      case Result(x) => Result(Left(x, p2))
      case NeedsInput(f) => p2 match {
        case Result(x) => Result(Right(x, p1))
        case NeedsInput(g) =>
          NeedsInput { s =>
            order(f(s), g(s))
          }
      }
    }

but if you define map, flatMap, etc these turn out to be ineffecient (left as an exercise to the reader).

Programs represented in "pull style" (the preceding are essentially "push style") are not so efficient to execute (in the obvious way):

sealed trait Event[A]
case class Instantly[A](x: A) extends Event[A]
case object Input extends Event[String]
case class Computation[A,B](ev: Event[A], f: A => B) extends Event[B]
case class Chaining[A](ev: Event[Event[A]]) extends Event[A]
case class Ordering[A1,A2](ev1: Event[A1], ev2: Event[A2]) extends Event[Either[(A1,Event[A2]),(A2,Event[A1])]]

Wouldn't I love to be able to define

def compile[A](ev: Event[A], cc: ): Program[A] = ???

but I haven't thought how to do that.

What gives? Why is it so hard to achieve both the memory managing benefits of pull programs and the lack of wasted computation of push programs?

This is still puzzling me.

Monday, May 27, 2013

Redo-style signals

Let's say I have signals a, b, c, d with dependencies

Inputs ~> a ~> b ~> c ~> d ~> Real World

The signal 'a' occasionally changes (due to inputs), which in turn changes the valid values of 'b', 'c' and 'd'. The value of 'd' is occasionally read and used to update some real-world object that the user can see. Maybe 'a' is set more often than 'd' is read; maybe it's the other way around; we're not sure.

The question is, when should the computations of 'b' 'c' and 'd' be performed? Should it be done in "push style", ie, every time 'a' changes, or "pull style", every time 'd' is read? In the first case we may waste a lot of computation if 'a' is set more frequently than 'd' is read. In the second, we do not waste computation, but we require a full check of the dependency tree every time we read 'd', just to see if something has changed in the meantime.

Well it turns out that the conceptual solution to this problem was already solved by Avery Pennarun in his program redo:

Dependencies are tracked in a persistent .redo database so that redo can check them later. If a file needs to be rebuilt, it re-executes the whatever.do script and regenerates the dependencies. If a file doesn't need to be rebuilt, redo can calculate that just using its persistent .redo database, without re-running the script. And it can do that check just once right at the start of your project build.

redo is based on the following simple insight: you don't actually care what the dependencies are before you build the target; if the target doesn't exist, you obviously need to build it. Then, the build script itself can provide the dependency information however it wants; unlike in make, you don't need a special dependency syntax at all. You can even declare some of your dependencies after building, which makes C-style autodependencies much simpler.

A signal's value is either up to date or obsolete. When the value is requested, if it is up to date, it returns that value immediately, not checking the dependency tree. If it's obsolete, it does a full recalculation.

Then, when a source value like 'a' changes, it simply asks the question: Who is currently relying on the correct value of this signal? In the case of 'a', that would be 'b', so 'a' notifies 'b' that the value is now obsolete (but without providing the new value), who notifies 'c', who notifies 'd'. After that notification chain is sent, no one is relying on the correct value for anything, so the next time 'a' changes, nothing happens; no notifications are done.

There is another benefit to this system. Say the dependency tree actually looks like

  d
 / \
b   c
 \ /
  a

With push-style signals, every time 'a' changes, 'd' gets updated twice, which is wasteful. With pull-style signals, 'd', would be updated once, but a would be consulted twice every time d is read. Redo-style signals get the benefit of push-style signals without having to consult the whole tree at each update.

Code

Monday, May 6, 2013

Justices Alito and Kennedy on Monads vs Applicatives

From the oral arguments in Alleyne v United States (audio):

JUSTICE ALITO: Now, if you were defending a case involving drug weight and your client maintained that he or she had nothing to do with these drugs, how would you proceed? Your argument would be: They're not my drugs, but if they were my drugs, they weren't -- they didn't weigh more than one kilo.

MS. MAGUIRE: Well, Justice Alito, those are strategical questions that come up in every trial case that we have. ... But those -- those strategic decisions exist whether or not the Court adopts this rule or doesn't adopt the rule.

...

JUSTICE KENNEDY: But -- but isn't it difficult for you to say he had nothing to do with the drugs, plus the drugs didn't weigh more than a certain amount?

MS. MAGUIRE: I don't believe that that is difficult, and I believe that those are decisions that you make in every case. ...

...

JUSTICE KENNEDY: Well, we're not getting very far with this. But one answer you could say is that, in order to preserve the constitutional right, you want us to have a bifurcated trial. I thought you were -- might say that.

MS. MAGUIRE: No, we are not -- we are not asking for a bifurcated trial. We are just asking that if there's one --

JUSTICE KENNEDY: That's good because that's an extra problem.

A bifurcated trial is one in which the jury first decides on a subset of questions, then the trial continues, and the jury later decides the remaining questions.

Alito notices that the trial is set to proceed in applicative style, and sees that the defense will be in an awkward position, since they must prove facts about an incident they claim never took place. Kennedy then offers a bid for a monadic trial, where the result of the first verdict can drive the argument leading to the second. But Ms. Maguire resists, for the same reason Kennedy had in mind: in law as in programming, monadic style brings extra costs.

The cost of a bifurcated trial is that you must interrupt the trial, wait for the jury to deliberate, then wait for the lawyers to adapt their strategies to the verdict, wherein either the lawyers have planned for both outcomes (wasting effort) or the trial is delayed further. Analogously, if you have a monadic query library, every use of >>= aka flatMap:

val X : Column[Bool]
val Y : Column[Int]
val Z : Column[Int]

X flatMap { x =>
    if (x)
        Y
    else
        Z
}

requires the preliminary query to complete, return control and data back to the main program, execute some code, make a new query, send it back to the database again, and resume. And the query optimizer doesn't get the benefit of seeing the whole query at once. This is why LINQ is not strictly monadic (and can operate on expressions, not just higher order functions) and all monadic query languages have some really awkward spots where they try to avoid this problem.

The analogy to speculative planning by the lawyers would be speculative execution of the closure { x => ...} by the client environment, which is theoretically possible but I am aware of no software that actually does this.

(In this case it would seem that even if the court holds that the trial should proceed applicatively, Ms Maguire's client gets the benefit of a monadic trial because he started his appeal after the primary verdict had already been reached; but no other defendant to whom the ruling applies will get this accidental benefit. Perhaps this explains why Ms Maguire seems relatively unconcerned by the issue).

Tuesday, April 30, 2013

Classical Fictions

It seems there is a fundamental tension between exploiting the full power of a constructive type theory and upholding the fictions of classical mathematics.

Noam Zeilberger

The desire not to be anti-classical puzzles me, because in my opinion the law of the excluded middle isn't just unprovable in automated proof assistants, it is really wrong; The statement (in Agda syntax):

∀ {p : Set} → p ⊎ (¬ p)

is usually read "for all statements p, p is true or p is false" but a more faithful translation (to the way the language actually behaves) would be "There exists a program, that, when given any statement, either proves that it is true or proves that it is false." Such a program is famously impossible.

But more than that, even if we could invoke the hand of God and introduce such a program as a postulate, we would still have an impossibility, since Gödel showed that there must be statements that can neither be proven true nor false -- in other words, the constructed law of the excluded middle is impossible even to God. Such a postulate isn't just wrong, it is one of the most widely known wrong things in math.

So it seems to me, that any language that is not anticlassical must be missing the benefit of a few assumptions. I would like to see

¬ (∀ {p : Set} → p ⊎ (¬ p))

to be provable, translated as "If you show me a program and claim it can prove or disprove any statement, I will analyze that program and find a case on which it fails."

The implementation of the above may involve pattern matching on functions (ie, inspecting to see what parts it is built of (lambda, application, constructor, ...)), which would violate extentionality, which I have no problem with because I think extentionality is also obviously untrue. Sure, you may define a

data ExtentionallyEqual {A B : Set} (f : A → B) (g : A → B) : Set where
    ext-equal : (pf : ∀ {a} → f a ≡ g a) → ExtentionallyEqual f g

and reason about it (as you may construct a classical theory within an anticlassical language), but there's no need for all terms in the language to conform, especially when there exist extentionally equal terms with wildly different performance characteristics.

Plus, if you could pattern match on functions, then the compiler could be written as an ordinary function in the source language, in the same execution environment; there would be no need to treat compilation as a "magical" task that happens outside of the main program. And much of the distinction between macros and functions disappears, as all terms are open for inspection. Of course, the inspections would need to respect whatever level of equality the language generally aims to support, and I am thinking but I haven't thought about it too deeply that in Agda that would mean that all pattern matching on functions must only see closed normal forms (which is not unlike how pattern matching already behaves in Agda).

Saturday, April 20, 2013

Axiomatic FRP

OK so there's this thing called FRP

Functional Reactive Programming
Building programs that vary in time out of Signals and Events
Signals, Events
these are objects that you can work with, and you have a set of combinators for making new signals out of old signals, etc,

And existing implementations -- I haven't looked at every existing implementation, but the ones I have looked at -- either don't allow you to create Signals dynamically (ie all your signals are there at program initialization), or have problems with memory management.

and the problems with memory management come because you have Signals implemented in terms of callbacks, which must use weak pointers, so, even though you can't technically overflow the heap this way, you're relying on the underlying garbage collector to stop active behavior -- to stop events from firing.

So I was thinking, we can do better -- one way we can do better, at least -- is if we drop Signals and Event streams for now and focus on one-shot events, because these have a definite expiration attached to them -- once the one-shot has fired you no longer need to hold on to it, so that should improve the memory situation a little bit.

And in addition to that, we can write down a set of axioms that are obviously true of events, so that if we target an implementation to the axioms in a minimal way, we can limit ourselves to only necessary screw-ups, avoiding superfluous screw-ups not required be the axioms.

So, my first attempt goes,

     Events1.agda 

  1  -- https://lists.chalmers.se/pipermail/agda/2010/002484.html
  2  -- https://lists.chalmers.se/pipermail/agda/2010/002485.html
  3  {-# OPTIONS --no-positivity-check #-}
  4  {-# OPTIONS --no-termination-check #-}
  5  {-# OPTIONS --type-in-type #-}
  6  
  7  -- Compiled against standard library 0.6
  8  module Events1 where
  9  
 10  open import Data.Sum using (_⊎_; inj₁; inj₂)
 11  open import Data.Product using (_×_; proj₁; proj₂; _,_)
 12  
 13  record EventSystem : Set where
 14    infixl 5 _map_
 15    field
 16      -- The type of a one-shot event, ie it does not repeat.
 17      Event : (result-type : Set) → Set
 18  
 19      -- Axioms of Events
 20  
 21      -- Basic map.
 22      _map_ : ∀ {r₁ r₂} (ev : Event r₁) (f : r₁ → r₂) → Event r₂
 23  
 24      -- Wait for the latter of one or two events.
 25      chain : ∀ {r} (ev : Event (r ⊎ Event r)) → Event r
 26  
 27      -- For any two events, one of them happens first.
 28      order : ∀ {r₁ r₂} (ev₁ : Event r₁) (ev₂ : Event r₂)
 29            → Event ((r₁ × Event r₂) ⊎ (r₂ × Event r₁))

and I believe -- and you can look at these and tell me if I'm right or not -- but I believe that these are obviously true -- in a single-threaded, non-relativistic world. The order axiom is obviously not true in a multi-threaded world, where in waiting for the first of two events you may miss the second.

So. The first thing I notice, is that we're very close to having a monad, except we're missing pure, and chain aka join has that funny union type.

Well, I had a very good reason for not including pure. When I was writing this I was very worried about having a way to create an event in a side-effectless manner, which is what pure is. I wanted every event to be grounded to another event, so you never had events as pure objects just floating around. I even considered a cojoin-like definition

     main 

  1  instantly : ∀ {r} → Event r → Event (Event r)

except that that turned out not to be very useful for writing actual programs.

But anyway, I remembered after writing this that applicatives don't need pure, and you can see how you can use that same exact trick for monads, so we can forgive ourselves for writing it like this:

  1  record EventSystem : Set where
  2    infixl 5 _map_
  3    field
  4      Event : (result-type : Set) → Set
  5  
  6      instantly : ∀ {r} (x : r) → Event r
  7  
  8      _map_ : ∀ {r₁ r₂} (ev : Event r₁) (f : r₁ → r₂) → Event r₂
  9  
 10      chain : ∀ {r} (ev : Event (Event r)) → Event r
 11  
 12      order : ∀ {r₁ r₂} (ev₁ : Event r₁) (ev₂ : Event r₂)
 13            → Event ((r₁ × Event r₂) ⊎ (r₂ × Event r₁))

Now, honestly, I like the first rules better. The first rules are obviously true, in my opinion, whereas with the second rules we have to do more self-convincing. But monads are ubiquitous and there's no use in fighting a monad when you have a monad, so we go with the monadic Event, plus the order axiom.

(Though we should remember, if we're ever going to implement these axioms, the fact that we had to use a disjoint union to convince ourselves that instantly is valid is a clue that we will need to special-case constant signals).

Now these axioms, despite what I still insist is there obvious trueness, have a serious problem. Let's say I have two events

  2  x : Event ⊤
  3  y : Event ⊤

and I use the Event axioms to derive

  4  xy = chain (x map (λ _ → y))
  5  yx = chain (y map (λ _ → x))

which intuitively mean "wait for x to happen, then wait for y to happen", and "wait for y to happen, then wait for x to happen." Now one of these must hang. Either x comes first or y comes first, and if x comes first and you wait for y, then wait for x, the second waiting will never occur.

Now I claim that this does not falsify the Event axioms. An event that never returns is not illegitimate -- you could still have an event resulting in that never comes to pass in the same way you can have a function returning that goes into an infinite loop and never returns. It would result in if it ever resulted in anything.

So we have to ask ourselves a question. Is this a problem worth fixing? Because if we're going to fix it, we're going to have to introduce extra restrictions in the Event axioms AND, critically, it will be impossible to make Event a monad, since we used only monad combinators to derive a never-occurring event, so we'd be giving up on a good deal of friendly generality by giving up on the monad.

But if I just think back to how often my interactive programs hang up waiting for events in the wrong order, the answer is "a lot". This is actually a pretty common problem. This is actually something we could stand to benefit from in the Event laws. Here is an attempted fix:

     SafeEvents.agda 

  1  {-# OPTIONS --type-in-type #-}
  2  {-# OPTIONS --no-positivity-check #-}
  3  {-# OPTIONS --no-termination-check #-}
  4  
  5  module SafeEvents where
  6  
  7  open import Data.Sum using (_⊎_)
  8  open import Data.Product using (_×_)
  9  
 10  π : {T : Set} → (T → Set) → Set
 11  π {T} A = (t : T) → A t
 12  
 13  record EventSystem : Set where
 14    field
 15      -- Time is probably not going to be represented by a Double or Int. It's not a "time" so
 16      -- much as a handle for getting at events.
 17      Time : Set
 18  
 19      -- We don't really care what this means.
 20      min : Time → Time → Time
 21  
 22      Event : (A : Time → Set) → (t : Time) → Set
 23  
 24      -- An event that happens without delay.
 25      instantly : ∀ {t A} (x : π A) → Event A t
 26  
 27      -- This is like usual map, but be careful, you also get a time.
 28      _map_ : ∀ {t A B} (ev : Event A t) (f : ∀ {t} → A t → B t) → Event B t
 29  
 30      -- This represents waiting for the second of two events.
 31      chain : ∀ {A t} → Event (Event A) t → Event A t
 32  
 33      -- Total ordering: for any two events, one happens first, and the other second.
 34      order : ∀ {A₁ A₂ t₁ t₂} (ev₁ : Event A₁ t₁) (ev₂ : Event A₂ t₂)
 35            → Event (λ t → (A₁ t × Event A₂ t) ⊎ (A₂ t × Event A₁ t)) (min t₁ t₂)

These axioms, I think, are not so obvious, so they require some explanation.

First, Event now takes a Time parameter in its type:

  6  Event : (A : Time → Set) → (t : Time) → Set

t is not the time at which the event happens; it is some time before the event happens. ie, an Event A t is an event that results in A after time t. So if t₁ comes before t₂, for example (a property not describable by these axioms), an event t₂ could also plausibly be an event t₁ -- though we have no axioms to get at this concept.

Now, instantly is written differently:

  7  instantly : ∀ {t A} (x : π A) → Event A t

The π is hiding some things; this could be written

  8  instantly : ∀ {t A} (x : (t : Time) → A t) → Event A t

In other words, the argument you give to instantly should be a function taking a Time as its argument, and returning a result whose type depends on that time. I say "should" rather than "must" because you can see that this is not actually a restriction on the caller -- x is free to ignore the time and return the same result it would have anyway; this type of instantly is actually more powerful and less general than the old one.

Here is how _map_ turned out:

  9  _map_ : ∀ {t A B} (ev : Event A t) (f : ∀ {t} → A t → B t) → Event B t

There might be a more useful way to write map, one that gave f more access to the times, but I was worried that could create a problem, so I stuck pretty close to the usual _map_ definition.

But chain holds the difference from the old Event laws, and this is more restrictive on the caller:

 10  chain : ∀ {A t} → Event (Event A) t → Event A t

Some of the parameters have been eta-removed, making it look suspiciously monadic, but this can be written as:

 11  chain : ∀ {A t} → Event (λ t′ → Event A t′) t → Event A t

In other words, you can chain an event producing an event provided that the inner event is constructed based on any time. This is giving me some kind of hints that times need types associated with them -- maybe Time actually represents an observable object -- but I'm not sure what to do about that right now.

And lastly, order gives us the kind of useful information needed to call chain:

 12  order : ∀ {A₁ A₂ t₁ t₂} (ev₁ : Event A₁ t₁) (ev₂ : Event A₂ t₂)
 13        → Event (λ t → (A₁ t × Event A₂ t) ⊎ (A₂ t × Event A₁ t)) (min t₁ t₂)

And we can verify that the necessary information is contained in order by writing a second function to get the second of two events:

 14  second : ∀ {r t₁ t₂} → Event r t₁ → Event r t₂ → Event r (min t₁ t₂)
 15  second ev₁ ev₂ = chain (order ev₁ ev₂ map λ {
 16           (inj₁ (r , next)) → next;
 17           (inj₂ (r , next)) → next
 18      })

And that works. And now we have time-safe events.

Wednesday, April 17, 2013

Applicatives do not need `pure`

A few days ago I said:

Usually, applicatives are assigned a third power, which is the power to take a plain object and bring it into the functor in the simplest way possible, eg 2 becomes [ 2 ]. This power is essential to writing generic, recursive programs with applicatives.

I take it back. I now believe that nothing interesting you can do with an applicative functor requires `pure`.

Assumption: if the only practical difference between two programs is that one applies pure at the very end, it is no more interesting than the other.

For example, pure 3 is not interesting, because it doesn't tell you anything that 3 doesn't, and (+) <$> (pure x) <*> (pure y) is not interesting because it is equivalent to pure (x + y).

I had been thinking of ways to represent events in FRP without bringing in memory issues that usually come with FRP. This meant I was spending a lot of time looking at different algebraic structures, and trying to peel off the assumptions that weren't needed. At one point I had a structure like the following:

     UnpureApplicatives.agda 

  1  module UnpureApplicatives where
  2  
  3  open import Level
  4  
  5  -- Here is a definition of applicatives that does not include
  6  -- a `pure` axiom.
  7  
  8  record UnpureApplicative {l} (F : Set l → Set l) : Set (suc l ⊔ suc zero) where
  9    field
 10      -- The basic map axiom.
 11      _ap\_ : ∀ {A B} (f : A → B) (x : F A) → F B
 12  
 13      -- The combine axiom.
 14      _/ap\_ : ∀ {A B} (f : F (A → B)) (x : F A) → F B

which has map and ap (as all applicatives do) but lacks pure, and provides no way of defining pure.

But we can still define something that is just as good as sequence:

     Test1.agda 

  1  open import Level
  2  open import UnpureApplicatives
  3  
  4  module Test1 {l} (F : _) (applicative : UnpureApplicative {l} F) where
  5      open import Data.List using (List; []; _∷_)
  6      open import Data.Sum using (_⊎_; inj₁; inj₂)
  7  
  8      open UnpureApplicative applicative
  9  
 10      -- We can derive this from the other two.
 11      -- Agda auto was actually able to solve this one.
 12      _/ap_ : ∀ {B} {A : Set l} (f : F (A → B)) (x : A) → F B
 13      _/ap_ f x = (λ z → z x) ap\ f
 14  
 15      -- This is how sequence now looks.
 16      -- It's ugly, but it gets the job done.
 17      sequence : ∀ {A} → List (F A) → List A ⊎ F (List A)
 18      sequence [] = inj₁ []
 19      sequence (x ∷ list) with sequence list
 20      ...                     | inj₁ plain = inj₂ ((_∷_ ap\ x) /ap plain)
 21      ...                     | inj₂ appy = inj₂ ((_∷_ ap\ x) /ap\ appy)

and the lack of pure is not "interesting" here because it may be applied at the end, by pattern matching on .

So what we've really done is use this "unpure" applicative to build a regular "pure" applicative:

     BuildFromUnpure.agda 

  1  open import UnpureApplicatives
  2  
  3  module BuildFromUnpure {l} (F : Set l → Set l) (Unpure : UnpureApplicative F) where
  4    open import Data.Sum using (_⊎_; inj₁; inj₂)
  5    open UnpureApplicative Unpure using (_ap\_; _/ap\_)
  6  
  7    F' : Set l → Set l
  8    F' A = A ⊎ F A
  9  
 10    pure : ∀ {A} → A → F' A
 11    pure = inj₁
 12  
 13    _/ap'\_ : ∀ {A B} (f : F' (A → B)) (x : F' A) → F' B
 14    inj₁ f /ap'\ inj₁ x = inj₁ (f x)
 15    inj₁ f /ap'\ inj₂ x = inj₂ (f ap\ x)
 16    inj₂ f /ap'\ inj₁ x = inj₂ ((λ f → f x) ap\ f)
 17    inj₂ f /ap'\ inj₂ x = inj₂ (f /ap\ x)

And I think it is clear that these two definitions are equally interesting, and, if a definition for pure were supplied originally, it could always be applied at the end, and so would add nothing.

Also, you could take an unpure applicative, make it a regular applicative, then drop the pure, and have something equivalent to the original unpure applicative.