from the desk of brilliant paula bean
implicit class StreamOfResults[T](source: Stream[Result[T]]) { def |>[U](op: T ⇒ Result[U]) = source map (t ⇒ t flatMap op) def filter(p: T ⇒ Outcome) = source |> (x => p(x) andThen Good(x)) def map[U](f: T ⇒ U) = source map (_ map f) }
implicit class StreamOfResults[T](source: Stream[Result[T]]) { def |>[U](op: T ⇒ Result[U]) = source map (t ⇒ t flatMap op) def filter(p: T ⇒ Result[_]) = source |> (x ⇒ p(x) returning x) def map[U](f: T ⇒ U) = source map (_ map f) }
E.g. use case:
// this method could be written in a couple of lines, but it's better to keep the details def streamOfNewEobSelectors(): StreamOfResults[Element] = { // this function pairs an html element with its content def withContent(e: Element): Result[(Element, String)] = e.outerHtml() map ((e, _)) // here we have a stream of elements paired with their content val pairs: StreamOfResults[(Element, String)] = streamOfEobElements |> withContent // here we filter the stream, leaving only the elements containing new stuff // note that the stuff we don't need is not kicked out, it's labeled as bad with an explanation val newOnes: StreamOfResults[(Element, String)]] = pairs filter (p => isNewClaim(p._2)) // here we forget the html newOnes map {case p:(Element, String) => p._1} }
Note that
filter
does not take a boolean
, it takes an Outcome
, which is a logical value, from the logic that I'm trying to work on. It's not Boolean.an amazingly funny piece of code
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala
So, spark gets a function to apply to data (e.g. in foreach, or in filter). Before running it, it (like a raccoon) cleans the function.
Enjoy the nice reading.
ClosureCleaner
So, spark gets a function to apply to data (e.g. in foreach, or in filter). Before running it, it (like a raccoon) cleans the function.
Enjoy the nice reading.
Entry tags:
fast scala programmers complaining about their keyboard
"S Ahmed
10:21 AM (51 minutes ago)
Reply to all
to scala-user
I have a 2010 MPB 17" i7 and after some time of development my keyboard really starts to heat up.
Do people with a more current MBP laptop have this same issue?
I think the only solution is to get an external keyboard.
The compiler is burning up my CPU :)
Thanks."
(the ring of fire, that's our Scala)
10:21 AM (51 minutes ago)
Reply to all
to scala-user
I have a 2010 MPB 17" i7 and after some time of development my keyboard really starts to heat up.
Do people with a more current MBP laptop have this same issue?
I think the only solution is to get an external keyboard.
The compiler is burning up my CPU :)
Thanks."
(the ring of fire, that's our Scala)
Entry tags:
multiversal equality in Scala
http://www.scala-lang.org/blog/2016/05/06/multiversal-equality.html
"The problems of universal equality in Scala are of course well known. Some libraries have tried to fix it by adding another equality operator with more restricted typing. Most often this safer equality is written ===. While === is certainly useful, I am not a fan of adding another equality operator to the language and core libraries. It would be much better if we could fix == instead. This would be both simpler and would catch all potential equality problems including those related to pattern matching."
"The problems of universal equality in Scala are of course well known. Some libraries have tried to fix it by adding another equality operator with more restricted typing. Most often this safer equality is written ===. While === is certainly useful, I am not a fan of adding another equality operator to the language and core libraries. It would be much better if we could fix == instead. This would be both simpler and would catch all potential equality problems including those related to pattern matching."
(between us)
The more I deal with Scala legislators, the more I agree that Scala is the new Java (and Java is the new Cobol, and Cobol is the new Latin, and Latin is the new petroglyph code).
A couple of weeks ago we went for a hike and saw a rather small dog, a male, that stood on its front paws and tried to pee on a trunk as high as possible.
Qed.
A couple of weeks ago we went for a hike and saw a rather small dog, a male, that stood on its front paws and tried to pee on a trunk as high as possible.
Qed.
dealing with time
Here's the essence of cache as I wrote about earlier tonight.
Note that you can override
trait TimeReader { def currentTime: Long = System.currentTimeMillis def flowOfTime = Iterator.iterate(currentTime){_ => currentTime} def interval(n:Long) = { flowOfTime.next // the first drop, may be stale val end = flowOfTime.next + n flowOfTime takeWhile (end >) } def interval(duration: Duration) = interval(duration.toMillis) //...
Note that you can override
currentTime
for {
value <- readFromSource
t <- interval(60 seconds)
} useThisValue(value)
e.g.
for {i <- 1 to 10; t <- interval(1 second) } {Thread.sleep(1); println(s"$i @$t") }
to shave a yak
Ok, my stuff is still not working. Discovered that for every imported row all kinds of hell are raised, including reading a "property" from db.
So I thought, wtf, let's use expiring cache, from spray. But spray for some reason is incompatible with the rest of the app, lift, etc. So I pulled the source code and decided to finish it with my own hands. Wtf, I wrote those expiring caches more than twice. Now what. Busy.
And then I figured, why map? All in need is an expiring value. So, "decorate" the property entries with caching. Stuff the property provider inside, give it TTL (time to live), and refresh on demand, storing it in a private
Wait, a
Ok, but what's
No, that's stupid. source is also a stream, and there's no need to pass it anywhere. This is how it should look:
But wait, so cache provides a flatMap, that's it right? And what does it do? It's a stream, but well... does it have a var? Probably not, just checks that the time is up, and then ends, meanwhile returning the same value all the time. What does it depend on? Time.
I want to mock time, how else do I check it all? So, I have to stuff the source of time, something like
Don't you see, time is also a stream.
What we produce, having a stream of time and a stream of values is pairs,
Of course the source of time is not infinite, it just lasts
In short, what do we have here? We have a Cartesian product of input source and time segments.
That's it.
Once I got it, the true nature of caching kind of became clear to me. Everything's illuminated. МПЯ, момент предельной ясности, как выражаются люди с туманом в голове.
Well, of course, additionally we can pass
So I thought, wtf, let's use expiring cache, from spray. But spray for some reason is incompatible with the rest of the app, lift, etc. So I pulled the source code and decided to finish it with my own hands. Wtf, I wrote those expiring caches more than twice. Now what. Busy.
And then I figured, why map? All in need is an expiring value. So, "decorate" the property entries with caching. Stuff the property provider inside, give it TTL (time to live), and refresh on demand, storing it in a private
var
.Wait, a
var
? Wtf, a variable? No way. Let's make it a stream, and use it in loops, like for { val value ← cached(source:Something, ttl: Duration) } doStuff(value)
Ok, but what's
source
? It was Future[T]
in Spray's ExpiringLruCache
, it was something like that in com.google.common.cache
that I was kind of developing with Bob Lee while at Google and while naïve like a boyscout.No, that's stupid. source is also a stream, and there's no need to pass it anywhere. This is how it should look:
for { val value0 ← (source:Stream[T]) // a stream val value ← cache(ttl: Duration)(value0) } doStuff(value)
But wait, so cache provides a flatMap, that's it right? And what does it do? It's a stream, but well... does it have a var? Probably not, just checks that the time is up, and then ends, meanwhile returning the same value all the time. What does it depend on? Time.
I want to mock time, how else do I check it all? So, I have to stuff the source of time, something like
for { val value0 ← (source:Stream[T]) // a stream val value ← cache(ttl: Duration, timeSource: TimeReader = StandardTime/*we have this class*/)(value0) } doStuff(value)
Don't you see, time is also a stream.
What we produce, having a stream of time and a stream of values is pairs,
(value:T, time:Timestamp)
, then we are free to abandon time
Of course the source of time is not infinite, it just lasts
ttl
units of time.In short, what do we have here? We have a Cartesian product of input source and time segments.
That's it.
Once I got it, the true nature of caching kind of became clear to me. Everything's illuminated. МПЯ, момент предельной ясности, как выражаются люди с туманом в голове.
Well, of course, additionally we can pass
doStuff
to the stream; it's a codata, is not it. Just not what I was going to talk about, but anyway.баловство со скалой
scala> sealed trait Nat defined trait Nat scala> case object Z extends Nat { override def toString = "0" } defined module Z scala> case class S(n:Nat) extends Nat { override def toString = (n.toString.toInt + 1).toString } defined class S scala> def pred(n:Nat) = n match { | case Z => Z | case S(x) => x | } pred: (n: Nat)Nat scala> pred(S(S(S(S(Z))))) res3: Nat = 3
Entry tags:
two variances in Scala
scala> val s = Set("a", "b", "c") s: scala.collection.immutable.Set[String] = Set(a, b, c) scala> val t = s.map(_ + ":)") t: scala.collection.immutable.Set[String] = Set(a:), b:), c:)) scala> val s = Set("a1", "a2", "a3") s: scala.collection.immutable.Set[String] = Set(a1, a2, a3) scala> val t = s.map(_ take 1) t: scala.collection.immutable.Set[String] = Set(a) scala> val u:Set[Any] = s map identity u: Set[Any] = Set(a1, a2, a3) scala> val v:Set[Any] = s <console>:8: error: type mismatch; found : scala.collection.immutable.Set[String] required: Set[Any] Note: String <: Any, but trait Set is invariant in type A. You may wish to investigate a wildcard type such as `_ <: Any`. (SLS 3.2.10) val v:Set[Any] = s ^
From a categorist's p.o.v, wtf, if we have
map
, we have a covariant functor. But wow, it's "type theory", covariance here means only covariance w.r.t. subtyping. So, big deal, map with identity, no? I mean, not being a typist, I don't even understand the problem. Do you?открываются мои глазыньки на sbt
Когда я читаю про includeFilter, я конкретно не понимаю. Они вообще в курсе, что такое фильтр? Ну и т.д.
Проблема с sbt в том, что там компьютерного саенса нету, и с fp не очень, а есть самопал и куча адхоков.
Я бы его переписал. С помощью scalaz, например.
Теоретик.
Проблема с sbt в том, что там компьютерного саенса нету, и с fp не очень, а есть самопал и куча адхоков.
Я бы его переписал. С помощью scalaz, например.
Теоретик.
generating build info
val buildInfoGenerator = taskKey[Seq[File]]("versionInfo") def buildInfo: String = { def stringLiteral(s: String) = "\"" + s.trim.replaceAll("\\n", "; ") + "\"" val gitInfo = stringLiteral(("git show --summary" !!) split "\\n" take 4 mkString "; ") s""" /* Build Information */ object BuildInfo { // (does not compile) val BuildDate = new org.joda.time.DateTime(${System.currentTimeMillis()}L) val BuildDate = new java.util.Date(${System.currentTimeMillis()}L) val BuildNumber = ${sys.env.get("BUILD_NUMBER") map stringLiteral} val BuildId = ${sys.env.get("BUILD_ID") map stringLiteral} val Hostname = ${stringLiteral("hostname" !!)} val GitInfo = $gitInfo val GitBranch = ${stringLiteral("git rev-parse --abbrev-ref HEAD" !!)} } """ } def generate(folder:File): Seq[File] = { println(s"Generating in $folder") val file = folder / "BuildInfo.scala" val contents = buildInfo IO.write(file, contents) Seq(file) } buildInfoGenerator in Compile := generate((sourceManaged in Compile).value) sourceGenerators in Compile <+= (buildInfoGenerator in Compile).toTask
The thing is, I could not yet figure out how to include this into project(s).