Montag, 24. Juni 2013
Freitag, 31. Mai 2013
Calling Idris from Javascript and vice-versa.
Until now the JavaScript backend for Idris has been a nice little gimmick to play around with, but when it comes to writing real applications it's pretty limited.
The limitations originates from the way how the FFI interacts with JavaScript. Until now it was not possible to call Idris code from within JavaScript which is essential when you want to register callbacks. In this blogpost I present my current work on the Idris FFI and JavaScript backend which enables this functionality. (Until these features have been merged into master you can track the development here)
Let's write a very simple piece of code that just changes a text label when it gets
clicked.
To do this we will need to define our onclick function. It'll look like this:
The second argument of our onclick function is of type "FFunction" takes an FUnit and returns a FAny (IO ()). mkForeign translates this into a function of type () -> IO (). For more information about the Idris FFI take a look at the tutorial. I've got a little bit of code on my gist that shows the FFI in action.
Have fun with calling functions back and forth :3
The limitations originates from the way how the FFI interacts with JavaScript. Until now it was not possible to call Idris code from within JavaScript which is essential when you want to register callbacks. In this blogpost I present my current work on the Idris FFI and JavaScript backend which enables this functionality. (Until these features have been merged into master you can track the development here)
Let's write a very simple piece of code that just changes a text label when it gets
clicked.
To do this we will need to define our onclick function. It'll look like this:
setOnClick : HTMLElement -> (() -> IO ()) -> IO ()
setOnClick (Elem p) f =
mkForeign (FFun ".onclick=" [FPtr, FFunction FUnit (FAny (IO ()))] FUnit) p f
The second argument of our onclick function is of type "FFunction" takes an FUnit and returns a FAny (IO ()). mkForeign translates this into a function of type () -> IO (). For more information about the Idris FFI take a look at the tutorial. I've got a little bit of code on my gist that shows the FFI in action.
Have fun with calling functions back and forth :3
Mittwoch, 13. Februar 2013
Shrinking Idris JavaScript with Closure
Compiling Idris to JavaScript can result in some pretty large files. In fact, small Idris programs can become so large that one might consider to fall back to JavaScript because it's just not worth it.
Let's take a look at a very simple example:
However, it gets even worse. The above program can be written in a different way. Let's give it a try.
Let's take a look at a very simple example:
main : IO ()
main = do
print l
where
l : List Nat
l = [1,2,3,4,5]
Let's compile this program and take a look at the size of the generated JavaScript.
raichoo@lain tmp» ls -lh test1.js
-rw-r--r-- 1 raichoo users 35K Feb 13 17:40 test1.js
35K of code is actually quite a lot for such a small program, but let's take into account that the file contains the whole runtime plus all needed parts of the standard library.
However, it gets even worse. The above program can be written in a different way. Let's give it a try.
module Main
main : IO ()
main = do
print l
where
l : List Nat
l = [1..5]
Just a small change. Instead of writing down the whole List we are using a range. Now let's take a look at the output.
raichoo@lain tmp» ls -lh test2.js
-rw-r--r-- 1 raichoo users 99K Feb 13 17:45 test2.js
Yikes, 99K! How did that happen?
Ranges are not built in, they are just a syntax extension that has been defined in the standard library and it uses a couple of functions to do what it does. We are now pulling in even more of the standard library.
Anyway, a simple change and the code grows by a factor of almost 3. Granted, there is an upper bound to the amount of code we can pull in from the library, but that's not very comforting.
Is there something we can do about this?
Yes. The Google Closure compiler can do a whole lot of optimizations on JavaScript. Apart from the usual minifying it also can do inlining and dead code elimination.
Running Closure on our JavaScript files yields the following results:
Ranges are not built in, they are just a syntax extension that has been defined in the standard library and it uses a couple of functions to do what it does. We are now pulling in even more of the standard library.
Anyway, a simple change and the code grows by a factor of almost 3. Granted, there is an upper bound to the amount of code we can pull in from the library, but that's not very comforting.
Is there something we can do about this?
Yes. The Google Closure compiler can do a whole lot of optimizations on JavaScript. Apart from the usual minifying it also can do inlining and dead code elimination.
Running Closure on our JavaScript files yields the following results:
raichoo@lain tmp» closure test1.js > test1-cl.js
raichoo@lain tmp» closure test2.js > test2-cl.js
raichoo@lain tmp» ls -lh test?-cl.js
-rw-r--r-- 1 raichoo users 20K Feb 13 18:12 test1-cl.js
-rw-r--r-- 1 raichoo users 63K Feb 13 18:13 test2-cl.js
That's smaller, but we can do even better. Idris targets Closure's advanced optimizations which can be enabled with the
Here's the result:
Now that's A LOT better. While Closure cannot get rid of the additional library code, it can eliminate code from Idris' runtime (we don't need big integers in this example, therefore Closure just get's rid of the code). Names get compressed and inlining takes place, etc etc.
I hope this shows that's Idris can create reasonably sized JavaScript files with a little help from it's friends.
Have fun with that!
--compilation_level=ADVANCED_OPTIMIZATIONS flag (e.g. closure --compilation_level=ADVANCED_OPTIMIZATIONS). We don't need to take care of the Closure Guidelines ourselves, Idris does that for us.Here's the result:
raichoo@lain tmp» ls -lh test?-cla.js
-rw-r--r-- 1 raichoo users 7.9K Feb 13 18:18 test1-cla.js
-rw-r--r-- 1 raichoo users 34K Feb 13 18:18 test2-cla.js
Now that's A LOT better. While Closure cannot get rid of the additional library code, it can eliminate code from Idris' runtime (we don't need big integers in this example, therefore Closure just get's rid of the code). Names get compressed and inlining takes place, etc etc.
I hope this shows that's Idris can create reasonably sized JavaScript files with a little help from it's friends.
Have fun with that!
Montag, 21. Januar 2013
Idris to JavaScript: Playing with the FFI
Until now the FFI for the JavaScript backend for Idris is quite primitive and somehow I'd like to keep it that way since I don't want to make invasive changes to the language just because of a different target. Therefore a started a little experiment to see how far one can get with the current FFI.
In JavaScript can have fields of objects, methods, functions, arrays etc etc. and we need to cover all these cases. Let's start with a little case study. We want to manipulate the DOM. Here is a little HTML snippet to begin with:
In this example I'm using the Ptr type for raw JavaScript values and wrap them in ADTs. The interesting part is in the definition of the foreign functions. Functions starting with "." are methods or fields, that means that the first argument of the function is handled as and object and the function call gets translated into a method or field name. Functions ending with "=" are turned into assignments
Another thing we have to consider is that sometimes we want to refer to a method with no arguments, therefore we have to distinguish them from reading a field. In our example we read the value of the field
Operations on arrays are declared like this:
Working with the FFI is still dangerous, I'm currently unaware of a different way to do this without changing Idris' FFI which is something I don't want to. Another thing I don't want to do is making the FFI overly complex, it should be very low level and provide the basic building blocks for interacting with JavaScript. Anyway, patches are welcome ^^
One thing that might be worth considering is a way to declare safe foreign calls that do not need to be wrapped in IO.
In JavaScript can have fields of objects, methods, functions, arrays etc etc. and we need to cover all these cases. Let's start with a little case study. We want to manipulate the DOM. Here is a little HTML snippet to begin with:
<html> <head> </head> <body> <div id="test">Foobar</div> <script src="test.js"></script> </body> </html>Now we want to replace the text of the Node with the id
test
module Main
data HTMLElement : Type where
Elem : Ptr -> HTMLElement
data NodeList : Type where
Nodes : Ptr -> NodeList
query : String -> IO NodeList
query q = do
e <- mkForeign (FFun "document.querySelectorAll" [FString] FPtr) q
return (Nodes e)
item : NodeList -> Int -> IO HTMLElement
item (Nodes p) i = do
i <- mkForeign (FFun ".item" [FPtr, FInt] FPtr) p i
return (Elem i)
getId : HTMLElement -> IO String
getId (Elem p) = mkForeign (FFun ".id" [FPtr] FString) p
setText : HTMLElement -> String -> IO ()
setText (Elem p) s =
mkForeign (FFun ".textContent=" [FPtr, FString] FUnit) p s
main : IO ()
main = do
e <- query "#test"
i <- item e 0
s <- getId i
setText i (s ++ ": SUPERFOO!!!")
In this example I'm using the Ptr type for raw JavaScript values and wrap them in ADTs. The interesting part is in the definition of the foreign functions. Functions starting with "." are methods or fields, that means that the first argument of the function is handled as and object and the function call gets translated into a method or field name. Functions ending with "=" are turned into assignments
Another thing we have to consider is that sometimes we want to refer to a method with no arguments, therefore we have to distinguish them from reading a field. In our example we read the value of the field
id. If we wanted to turn that into a method call we need to declare it like this:mkForeign (FFun ".id" [FPtr, FUnit] FString) p ()
We simply add an argument of type FUnit to the argument list and apply ().Operations on arrays are declared like this:
mkForeign (FFun ".id[]" [FPtr, FInt] FString)
mkForeign (FFun ".id[]=" [FPtr, FInt, FString] FString)
The second argument is treated as an indexWorking with the FFI is still dangerous, I'm currently unaware of a different way to do this without changing Idris' FFI which is something I don't want to. Another thing I don't want to do is making the FFI overly complex, it should be very low level and provide the basic building blocks for interacting with JavaScript. Anyway, patches are welcome ^^
One thing that might be worth considering is a way to declare safe foreign calls that do not need to be wrapped in IO.
Freitag, 18. Januar 2013
Towards dependently typed webprogramming with Idris
JavaScript pretty much has become the lingua franka of the web. It runs in the browser, and since nodejs it's also possible to write server applications. However, it lacks a powerful type-system, and type-systems are something a lot of us have come to love. Anyway, this is not the time for the usual "dynamic vs. static" and "weak vs. strong typing" blog post. Today I want to write about a project I've started working on which brings dependent types to the JavaScript ecosystem meaning a JavaScript backend for the Idris compiler.
The foundation of this effort, the programming language Idris by Edwin Brady. Idris is basically the answer to the question: "What would Haskell look like if it had full dependent types?" It has strict semantics but also offers annotations for non-strict evaluation. If you want to know more about Idris can highly recommend the tutorial. Sadly there is not a lot of material out there to learn about programming with dependent types but I will link some resources in this blog post.
The backend itself compiles a large set of the language into a self contained Javascript file and offers a simple FFI to communicate with other libraries. It's still every experimental and I would not recommend to use it in your dayjob. However, if you feel like it feel free to contribute libraries and patches to complete the experience and transform Idris into a suitable language for web programming :)
How to use it:
Let's write a little Idris program and compile it to JavaScript:
Now lets use the FFI to call into JavaScript land!
This is just a little fraction of what you can do with the backend. Currently there is no support for types like Word8 and Word16 because it would not really make sense in a JavaScript context. There is also no support for filesystem operations. The code that gets generated is still rather large but with the google closure compiler you can reduce the size by a factor of 2 (advanced compilation is not supported at the moment).
References:
The foundation of this effort, the programming language Idris by Edwin Brady. Idris is basically the answer to the question: "What would Haskell look like if it had full dependent types?" It has strict semantics but also offers annotations for non-strict evaluation. If you want to know more about Idris can highly recommend the tutorial. Sadly there is not a lot of material out there to learn about programming with dependent types but I will link some resources in this blog post.
The backend itself compiles a large set of the language into a self contained Javascript file and offers a simple FFI to communicate with other libraries. It's still every experimental and I would not recommend to use it in your dayjob. However, if you feel like it feel free to contribute libraries and patches to complete the experience and transform Idris into a suitable language for web programming :)
How to use it:
Let's write a little Idris program and compile it to JavaScript:
module Main
product : List Integer -> Integer
product = foldl (*) 1
fac : Integer -> Integer
fac n = product [1..n]
main : IO ()
main = print (fac 300)
and compile it> idris --target javascript test.idr -o test.js
and run it> node test.js
… long output is long …
As you can see, the backend also supports big integers. The generated file is completely self contained including code from the standard library like typeclasses etc. (not everything but the code you need).Now lets use the FFI to call into JavaScript land!
module Main
product : List Integer -> Integer
product = foldl (*) 1
fac : Integer -> Integer
fac n = product [1..n]
alert : String -> IO ()
alert s = mkForeign (FFun "alert" [FString] FUnit) s
main : IO ()
main = alert $ show (fac 300)
Compile and run it in a browser of your choice ^^This is just a little fraction of what you can do with the backend. Currently there is no support for types like Word8 and Word16 because it would not really make sense in a JavaScript context. There is also no support for filesystem operations. The code that gets generated is still rather large but with the google closure compiler you can reduce the size by a factor of 2 (advanced compilation is not supported at the moment).
References:
- Certified Programming With Dependent Types
- Talk by Andreas Bogk at Chaos Communication Camp 2011
- Agda Wiki
Samstag, 4. Februar 2012
Setting up a Tor Bridge Relay on illumos
Times are getting rough again. There are regimes out there that censor the ability of their citizens to express themselves or communicate with each other, sometimes even lives are at stake. Communication isn't a luxury, it's what makes us human, it empowers us, tears down walls between people and helps us to understand each other. Censorship and total surveillance is a violation of human rights, it's something we need to fight.
One way of doing this is to support the Tor Project. In this blogpost I will show how to set up a so called bridge relay on illumos, an entry point to the tor network which empowers people that suffer under the influence of censorship and surveillance to access the internet.
Step 1: Building Tor under illumos
Building Tor is easy as pie. All you need is the libevent src and the Tor src.
Step 2: Setting up an illumos zone
Since illumos inherits all the awesome features of OpenSolaris we can isolate our Tor bridge inside of a zone. We will create a zone with the name "tor".
Before we start we need to create a zfs dataset for our zone. I usually put mine in /export/zones (which itself is a dataset) like so:
Step 3: Setup TOR
Time to get serious. After building Tor and putting it into our zone we now need to configure Tor to function as a Bridge Relay. Here we set up our bridge to listen on port 443. Since Tor traffic looks a lot like SSL it's a good place to run. Our
I recommend that you set up a tor user to avoid running as root. The problem is that you cannot run run a server on a privileged port when you are a mere user. We can use RBAC to give the tor user a profile that allows it to run services on such ports.
If you want to know more about the Tor Project, I recommend you this talk:
Step 1: Building Tor under illumos
Building Tor is easy as pie. All you need is the libevent src and the Tor src.
Step 2: Setting up an illumos zone
Since illumos inherits all the awesome features of OpenSolaris we can isolate our Tor bridge inside of a zone. We will create a zone with the name "tor".
Before we start we need to create a zfs dataset for our zone. I usually put mine in /export/zones (which itself is a dataset) like so:
[root@lain:~]> zfs create rpool/export/zones/tor[root@lain:~]> zonecfg -z tortor: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:test> create
zonecfg:test> set zonepath=/export/zones/torzonecfg:test> verify
zonecfg:test> commit
zonecfg:test> exit
[root@lain:~]> zoneadm list -cv
ID NAME STATUS PATH BRAND IP
0 global running / native shared
- test configured /export/zones/tor ipkg shared [root@lain:~]> zoneadm -z tor install[root@lain:~]> zoneadm -z tor boot
[root@lain:~]> zlogin -C torStep 3: Setup TOR
Time to get serious. After building Tor and putting it into our zone we now need to configure Tor to function as a Bridge Relay. Here we set up our bridge to listen on port 443. Since Tor traffic looks a lot like SSL it's a good place to run. Our
torrc should look like this:SocksPort 0
ORPort 443
BridgeRelay 1
Exitpolicy reject *:*I recommend that you set up a tor user to avoid running as root. The problem is that you cannot run run a server on a privileged port when you are a mere user. We can use RBAC to give the tor user a profile that allows it to run services on such ports.
[root@tor:~]> usermod -K defaultpriv=basic,net_privaddr tor
[tor@tor:~]> pfexec tor -f torrcIf you want to know more about the Tor Project, I recommend you this talk:
Mittwoch, 4. Januar 2012
Building GHC under Illumos
I like the bleeding edge, that's why I always want to have the newest compilers for my favorite languages and Haskell is one of those languages. In this blogpost I'm going to show you how to build the latest and greatest GHC on the Illumos distribution OpenIndiana. If you don't want to build it yourself you can pick up a fairly recent version of GHC from the SFE Repository.
To make things easier let's get our hand on a GHC binary from which we can bootstrap our environment, luckily there is a package for GHC 7.0.3 for Solaris which works just fine under Illumos. Just install it and put it in your PATH environment variable.
Due to it's Solaris heritage, Illumos currently ships with a pretty outdated version of GCC. I recommend that you install the GCC provided by the SFE Repository since the shipped version causes some linking problems when dealing with the hidden attribute. If you have to use the old compiler you can get rid of this issue by using this quick and dirty patch which just gets rid of some pragmas and macros:
diff --git a/includes/Rts.h b/includes/Rts.h
index 91ec76d..adbbe54 100644
--- a/includes/Rts.h
+++ b/includes/Rts.h
@@ -52,7 +52,7 @@ extern "C" {
// with visibility "hidden" to hide them outside the RTS shared
// library.
#if defined(HAS_VISIBILITY_HIDDEN)
-#define RTS_PRIVATE GNUC3_ATTRIBUTE(visibility("hidden"))
+#define RTS_PRIVATE //GNUC3_ATTRIBUTE(visibility("hidden"))
#else
#define RTS_PRIVATE /* disabled: RTS_PRIVATE */
#endif
diff --git a/rts/BeginPrivate.h b/rts/BeginPrivate.h
index 6471b92..1af4c90 100644
--- a/rts/BeginPrivate.h
+++ b/rts/BeginPrivate.h
@@ -6,5 +6,5 @@
error: visibility attribute not supported in this configuration; ignored
*/
#if defined(HAS_VISIBILITY_HIDDEN) && !defined(freebsd_HOST_OS)
-#pragma GCC visibility push(hidden)
+//#pragma GCC visibility push(hidden)
#endif
diff --git a/rts/EndPrivate.h b/rts/EndPrivate.h
index 4cfb68f..c2e3154 100644
--- a/rts/EndPrivate.h
+++ b/rts/EndPrivate.h
@@ -1,3 +1,3 @@
#if defined(HAS_VISIBILITY_HIDDEN) && !defined(freebsd_HOST_OS)
-#pragma GCC visibility pop
+//#pragma GCC visibility pop
#endif
Now we are good to go. Get your hands on some fresh tarball, perhaps the latest and greatest RC. To configure your future GHC just run something like:
./configure --prefix=~/Library/GHC/version --with-gmp-includes=/usr/include/gmp
~/Library/program/version. From here I can manage everything using softlinks but feel free to use any other prefix.
Be sure to run the GNU Make (gmake) command and not the Illumos one. Sadly gmake seems to be unable to handle multiple jobs when compiling GHC so our final step looks like this:
gmake && gmake install
From here on you should be able to build pretty much every version you wish for. If you pick a version that is supported by cabal-install installing additional packages from hackage is a piece of cake. I usually have more than one GHC installed, the one that is shipped with the current Haskell Platform and the latest version with all the hot new features.
Illumos has the potential to be one of the best platforms for developing Haskell since it has exquisite DTrace support. If you want to help to improve the Haskell experience on Illumos, feel free to contact me in #openindiana on freenode.
Mittwoch, 24. August 2011
More Fun with Scala "Dependent" Types
In my last post I wrote about lists that encode their length within their type and the possibilities that gives us to ensure correctness of our code. In this post I want to expand these ideas.
Let's assume that we have a list with an even number of elements. By knowing that the length is even we could do various things like extracting pairs. Odd length is not very suitable for this… no shit Sherlock!
To make this happen we first need boolean values in the type system to represent that a proposition is actually true and we need to be able to negate them.
In the next step we have to extend our type-level natural numbers. Let's give them a predecessor and a property that indicates if a number is even.
To make it a little nicer to read and write we will add a witness that represents that a natural number is even we check this with the
Now the fun part. We now start building our list with a getPairs method that only works on lists with an even length. Here is our interface for this NList:
The first case again is trivial, we are dealing with an empty list, the only thing that we can extract is an empty list.
Now this one is a little more tricky, we might know that a natural number
That's all. How does this look when we put it into action?
Very nice!
In future blogposts I'd like to show you how a language that bears these techniques in mind handles these problems.
Let's assume that we have a list with an even number of elements. By knowing that the length is even we could do various things like extracting pairs. Odd length is not very suitable for this… no shit Sherlock!
To make this happen we first need boolean values in the type system to represent that a proposition is actually true and we need to be able to negate them.
sealed trait Bool {
type Not <: Bool
}
sealed trait True extends Bool {
type Not = False
}
sealed trait False extends Bool {
type Not = True
}
In the next step we have to extend our type-level natural numbers. Let's give them a predecessor and a property that indicates if a number is even.
sealed trait Nat {
type IsEven <: Bool
type Pred <: Nat
}
sealed trait Z extends Nat {
type IsEven = True
type Pred = Z
}
sealed trait S[A <: Nat] extends Nat {
type IsEven = A#IsEven#Not
type Pred = A
}
To make it a little nicer to read and write we will add a witness that represents that a natural number is even we check this with the
=:= type constraint which witnesses that two types are equal.
sealed trait Even[A <: Nat]
object Even {
implicit def isEven[A <: Nat](implicit e: A#IsEven =:= True): Even[A] =
new Even[A]{}
}
Now the fun part. We now start building our list with a getPairs method that only works on lists with an even length. Here is our interface for this NList:
sealed trait NList[A <: Nat, +B] {
type Length = A
def getPairs(implicit e: Even[Length]): List[(B, B)]
def head: B
def tail: NList[Length#Pred, B]
}
The first case again is trivial, we are dealing with an empty list, the only thing that we can extract is an empty list.
case object NNil extends NList[Z, Nothing] {
def getPairs(implicit e: Even[Length]): List[(Nothing, Nothing)] =
Nil
def head = sys.error("empty list")
def tail = sys.error("empty list")
}
Now this one is a little more tricky, we might know that a natural number
A is even but what the typesystem does not know if A#Pred#Pred is also even when we call getPairs recursively. We now need to to create a witness that A#Pred#Pred is even when A is even. We can do this with another method that generates a witness when you hand it the witness for A's evenness, let's call that method predPredEven.
sealed case class NCons[A <: Nat, B](h: B, t: NList[A, B])
extends NList[S[A], B] {
private def predPredEven[A](e: Even[Length]): Even[Length#Pred#Pred] =
new Even[Length#Pred#Pred]{}
def getPairs(implicit e: Even[Length]): List[(B, B)] =
(head, tail.head) :: tail.tail.getPairs(predPredEven(e))
def head: B = h
def tail: NList[Length#Pred, B] = t
}
That's all. How does this look when we put it into action?
scala> val lo = NCons(1, NNil)
lo: nlist.NCons[nlist.Z,Int] = NCons(1,NNil)
scala> le.getPairs
res0: List[(Int, Int)] = List((1,2))
scala> NNil.getPairs
res1: List[(Nothing, Nothing)] = List()
scala> lo.getPairs
:12: error: could not find implicit value for parameter e: nlist.Even[lo.Length]
lo.getPairs
^
Very nice!
getPairs only works on lists with an even length, otherwise we get a compile error. Once again: If you want to play with this code check out this gist.
In future blogposts I'd like to show you how a language that bears these techniques in mind handles these problems.
Montag, 22. August 2011
A Taste of Dependent Types in Scala
One thing starts to cross my way very frequently is dependent typing. While I'm not an expert on this topic I found some very interesting things in various papers that I would like to be able to do in Scala. Dependent types help to verify that your program does what it's supposed to do by proving theorems.
Since Scala is not dependently typed we need to become a little tricky, but let's give it a try.
Think about the type signature of the following method called
By looking at the signature it should become obvious what the method does. You have a list
That's a pretty straight forward example, but there is a problem with this code. What if we do provide lists with different length? In one case we would not execute some functions in the other case we would lose data. So what can we do? We could throw an exception but that's already to late, our program is already in a state where things don't make sense anymore, we have to avoid this situation at all and the best thing to achieve this is that such code does not compile at all.
In a dependently typed language we could create a list that encodes it's length within the type of the list itself and write
The first thing we need is: natural numbers in the type system to encode the length of our list. That is easy:
Now let's start with the fun part, the list itself.
The first thing we need is the
We can now see that the type of
The next case is not as simple (if someone has a better idea of how to implement this, I would love to see it. Please drop a comment).
We can see that our
than it's tail which has length
Now let's put that little bugger to the test:
First lets define some
Zapping lists with the same length:
This works just fine, now lets see what happens if we
Exactly what we wanted! This code does not even compile since it doesn't make any sense.
This is a very basic example of what we can do with the Scala type system. Stay tuned for more!
If you would like to play around with the code check out my gist
Since Scala is not dependently typed we need to become a little tricky, but let's give it a try.
Think about the type signature of the following method called
zap (which stands for zip-apply):
def zap[A, B](l: List[A], fs: List[A => B]): List[B]
By looking at the signature it should become obvious what the method does. You have a list
l that provides elements of type A and a list fs that provides a function from A to B for each element in l. Lets look at the implementation:
def zap[A, B](l: List[A], fs: List[A => B]): List[B] =
(l, fs) match {
case (_, Nil) | (Nil, _) => Nil
case (a :: as, b :: bs) => b(a) :: zap(as, bs)
}
That's a pretty straight forward example, but there is a problem with this code. What if we do provide lists with different length? In one case we would not execute some functions in the other case we would lose data. So what can we do? We could throw an exception but that's already to late, our program is already in a state where things don't make sense anymore, we have to avoid this situation at all and the best thing to achieve this is that such code does not compile at all.
In a dependently typed language we could create a list that encodes it's length within the type of the list itself and write
zap in a way so that it would only accept lists of the same length. But since we are not in a dependently typed language we cannot do that, this concludes this blogpost, see you in another episode… ok, just kidding. We actually CAN do that in Scala. Here is how:
The first thing we need is: natural numbers in the type system to encode the length of our list. That is easy:
sealed trait Nat
sealed trait Z extends Nat
sealed trait S[A <: Nat] extends Nat
Z is our zero here and S[A <: Nat] is the successor of the natural number A, so we can create the natural numbers recursively: Z == 0, S[Z] == 1, S[S[Z]] == 2 etc.
Now let's start with the fun part, the list itself.
The first thing we need is the
trait for our NList.
trait NList[A <: Nat, +B] {
type Length = A
def zap[C](l: NList[Length, B => C]): NList[Length, C]
}
We can now see that the type of
zap enforces that the both lists need to have the same length A. We can now create our first case where we are zapping two lists of length 0 (or Z). The implementation is rather trivial.
case object NNil extends NList[Z, Nothing] {
def zap[A](l: NList[Length, Nothing => A]): NNil.type =
NNil
}
The next case is not as simple (if someone has a better idea of how to implement this, I would love to see it. Please drop a comment).
case class NCons[A <: Nat, B](head: B, tail: NList[A, B])
extends NList[S[A], B] {
def zap[C](l: NList[Length, B => C]): NList[Length, C] =
l match {
case NCons(f, fs) =>
NCons(f(head), tail zap fs)
}
}
We can see that our
NCons is actually a NList that is one element longer
than it's tail which has length
A, the pattern match inside our zap method only has to handle the cons case since the type signature does not allow anything else.
Now let's put that little bugger to the test:
First lets define some
NLists
scala> val l1 = NCons(1, NCons(2, NCons(3, NNil)))
scala> val l2 = NCons(1, NCons(2, NNil))
scala> val f = (_: Int) + 1
scala> val f1 = NCons(f, NCons(f, NCons(f, NNil)))
scala> val f2 = NCons(f, NCons(f, NNil))
Zapping lists with the same length:
scala> l1 zap f1
res1: nlist.NList[l1.Length,Int] = NCons(2,NCons(3,NCons(4,NNil)))
scala> l2 zap f2
res2: nlist.NList[l2.Length,Int] = NCons(2,NCons(3,NNil))
This works just fine, now lets see what happens if we
zap two lists with a different length:
scala> l1 zap f2
:14: error: type mismatch;
found : nlist.NCons[nlist.S[nlist.Z],Int => Int]
required: nlist.NList[l1.Length,Int => ?]
l1 zap f2
Exactly what we wanted! This code does not even compile since it doesn't make any sense.
This is a very basic example of what we can do with the Scala type system. Stay tuned for more!
If you would like to play around with the code check out my gist
Sonntag, 10. Juli 2011
From Functions to Monads in Scala
"Monad" that's a word you hear quite often these days. There is a lot of unjustified FUD whenever this word pops up. People seem to be downright scared about it. You can tell by the amount of rhetorical trick words people use to scare others away from trying to understand monads at all. "Academic", "no use in real world programming" etc etc. I find it quite frustrating to hear something like that even within the Scala community from time to time (even though this is getting rarer and rarer), since these are the "arguments" people bring up when they try to scare others away from Scala.
First of all, I'm not a category theorist. These things just fascinate me, and I enjoy reading about this in my free time. I'm your ordinary "Joe, the programmer" working at a small software company.
Still, I find these things useful in my day job. Why? Because as a programmer I'm building mathematical machines (namely software) all day long and these concepts help me to order my thoughts and make at easier to reason about my software and design stuff.
In this blogpost I want to show how to derive Monads from what you already know and give names to things you maybe never even thought about.
Categories
Since monads come from Category Theory we first have to define what a category is.
A category C basically consists of 2 sets ob(C) and hom(C) where we call ob(C) the objects of C and hom(C) the morphisms of C. Morphisms are maps that go between objects.
Never saw a Category in your every day programming? Sure? There is a Category called Hask, its objects are the Haskell types and its morphisms are Haskell functions. So you can think about that in a Scala context:
Let
There's more to a Category, we need to be able to compose morphisms and there has to be a special element called identity. Lets looks at composition first.
Let f and g be functions:
We can compose those 2 functions in Scala by using the
OK, that's half the rent. Now we just need the special element called identity which basically is a function that does nothing. It just takes a value and returns it unmodified. So when we
In short: The following 3 functions are equivalent (which means: when fed with the same input values they will yield the exact same result)
Now we know that a Category is (well, at least we know enough to start, a real category theorist can take you on a breathtaking journey from here :)). Let's move on!
Enter Higher-Kinded-Types
Next "scary word", but again: You use them all the time. First we need to know what a kind is. We know what types and values are. You can think of them as levels. A value like
OK, if we group values into types, can we group types? Yes, we can! We can group types into… *drumroll* KINDS! Now we have a third level and our world looks something like this:
In most languages we only deal with one kind called
We now have a basic idea of what kinds are, so what are higher kinded types? Like High-Order-Functions (functions that take functions as arguments and yield functions e.g. compose)
higher kinded types are type constructors that take type constructors as arguments and yield types (or maybe even type constructors).
"Wait a minute… are you talking about generics?"
Think about
So a higher kinded type looks something like this:
Functors
Now that we have higher kinded types, let's take a look at our functions
Now wouldn't it be cool if there was a morphism between those two categories, one that preservers composition and the identity rule? Something that could map a function of type
"Why is this useful?" you might ask. Just think about the kind of code reuse you could achieve. Let's look at
Let's check it out:
We could now reuse our function
The second ingredient we need to make our Functor complete is a morphism that maps a value of type
Endofunctors
Now, let's assume that we want something like this:
We want to use a function of type
What happened here? Remember how we mapped a function of type
This is a special kind of Functor that maps a category into itself and is called an Endofunctor
Natural Transformations
Now that we have morphisms between objects of a Category which are Scala functions and Functors which are basically morphisms between categories it's time to introduce morphisms between Functors.
Why do we need this? Well in the last example we ended up with an Endofunctor (we mapped a Category into itself). We need to get rid of one of the
Lets do a concrete example that involves
It happened again, but this time we ended up with a
This enables us to chain functions together in a very convenient way.
or a little more concise
Monads
This is actually what makes up a Monad. It's an Endofunctor with two Natural Transformations called unit and join (which is called
Conclusion
This is an introduction on how to think about monads and how to derive them from simple building blocks. They are a very powerful abstraction that pops up all the time. You use them all the time without knowing. Since programming is all about abstraction it's useful to recognize those underlying patterns. Don't let others fool you into believing that this is just some bizarre functional stuff that has no use in the real world. This is just the tip of the iceberg, an amazing journey lies ahead. Let's learn!
First of all, I'm not a category theorist. These things just fascinate me, and I enjoy reading about this in my free time. I'm your ordinary "Joe, the programmer" working at a small software company.
Still, I find these things useful in my day job. Why? Because as a programmer I'm building mathematical machines (namely software) all day long and these concepts help me to order my thoughts and make at easier to reason about my software and design stuff.
In this blogpost I want to show how to derive Monads from what you already know and give names to things you maybe never even thought about.
Categories
Since monads come from Category Theory we first have to define what a category is.
A category C basically consists of 2 sets ob(C) and hom(C) where we call ob(C) the objects of C and hom(C) the morphisms of C. Morphisms are maps that go between objects.
Never saw a Category in your every day programming? Sure? There is a Category called Hask, its objects are the Haskell types and its morphisms are Haskell functions. So you can think about that in a Scala context:
Let
f be a function of type Int => String. Int and String are both elements of ob(C) and f is an element of hom(C).
There's more to a Category, we need to be able to compose morphisms and there has to be a special element called identity. Lets looks at composition first.
Let f and g be functions:
f: A => B
g: B => C
We can compose those 2 functions in Scala by using the
compose method to get a new function.
g compose f // yields a function of type A => C
OK, that's half the rent. Now we just need the special element called identity which basically is a function that does nothing. It just takes a value and returns it unmodified. So when we
compose a function f with identity we get back a function that is equivalent to f and it should not matter if we compose f with identity or identity with f.
In short: The following 3 functions are equivalent (which means: when fed with the same input values they will yield the exact same result)
f // A => B
f compose id // A => B
id compose f // A => B
Now we know that a Category is (well, at least we know enough to start, a real category theorist can take you on a breathtaking journey from here :)). Let's move on!
Enter Higher-Kinded-Types
Next "scary word", but again: You use them all the time. First we need to know what a kind is. We know what types and values are. You can think of them as levels. A value like
1 and x => x (that's a function value, yes those are values too!) live a the value level while types like Int and Int => Int live at the type level. We group values into types. true and false are both values of type Boolean (think of types as sets of values).
OK, if we group values into types, can we group types? Yes, we can! We can group types into… *drumroll* KINDS! Now we have a third level and our world looks something like this:
kinds
---------
types
---------
values
In most languages we only deal with one kind called
* (pronounced type) that groups all types, but there are languages that allow infinite levels (I might cover one of them in future blog posts).
We now have a basic idea of what kinds are, so what are higher kinded types? Like High-Order-Functions (functions that take functions as arguments and yield functions e.g. compose)
higher kinded types are type constructors that take type constructors as arguments and yield types (or maybe even type constructors).
"Wait a minute… are you talking about generics?"
Think about
List. It's not a type by itself, you need to feed it a type as an argument to get a type. This is called a Type Constructor.List takes one type and returns a type, hence List has the kind * -> * (this notation is borrowed from Haskell). The Scala notation for a type constructor like List is List[_].
So a higher kinded type looks something like this:
trait Functor[F[_]] where F could be a List, Option or any other type constructor that takes one argument.Functors
Now that we have higher kinded types, let's take a look at our functions
f and g from the beginning and lets add a type constructor F[_] to the mix (this is Scala notation and means: takes one type and yields a type). We can now produce a whole new set of types with this higher kind (e.g. F[Int], F[String] etc.). We could now create functions that work with these types… hey I know this. This sounds like… a Category! Yes, it is! It's actually a Subcategory of our Hask Category we introduced at the beginning. Let's call this new Category D (so we don't confuse it with the type constructor F). So elements of ob(D) would be something like F[A] and for hom(D) it would be something like F[A] => F[B].
Now wouldn't it be cool if there was a morphism between those two categories, one that preservers composition and the identity rule? Something that could map a function of type
A => B to a function of type F[A] => F[B]? That's what's we call a Functor, it's a morphism between categories.
"Why is this useful?" you might ask. Just think about the kind of code reuse you could achieve. Let's look at
Option as a concrete example. It's a type constructor. So how do we map a function of type Int => Int to a function of type Option[Int] => Option[Int]? Can't you tell by now? The answer is the map method of Option ;).
Let's check it out:
scala> val f = (_: Int) + 1
f: (Int) => Int = <function1>
scala> Some(1) map f
res0: Option[Int] = Some(2)
We could now reuse our function
f with Option, there was no need to write a special version of f that works with Option.
The second ingredient we need to make our Functor complete is a morphism that maps a value of type
A to a value of type F[A]. In the case of Option this is just the factory function:
scala> Some(_: Int)
res0: (Int) => Some[Int] =
scala> Option(_: Int)
res1: (Int) => Option[Int] =
scala> Option(1)
res2: Option[Int] = Some(1)
scala> Some(1)
res3: Some[Int] = Some(1)
Endofunctors
Now, let's assume that we want something like this:
scala> Some(1) map {
| case x if x % 2 == 0 => Some("YES!")
| case _ => None
| }
res1: Option[Option[java.lang.String]] = Some(None)
We want to use a function of type
Int => Option[String] with Option's map method. But what's that? We get an Option[Option[String]]. That's stupid, now I have to unpack one Option to get the result I actually wanted.
What happened here? Remember how we mapped a function of type
A => B to a function of type F[A] => F[B] in the previous section about functors? What we did now is: we mapped a function of type Int => Option[String] to a function of Option[Int] => Option[Option[String]]. Yikes, we went a little bit over the top here, huh? We kind of like mapped something into an Option and into an Option this is how we ended up with an Option[Option[String]]. You can think of this as a result of a composition of 2 Option Functors.
This is a special kind of Functor that maps a category into itself and is called an Endofunctor
Natural Transformations
Now that we have morphisms between objects of a Category which are Scala functions and Functors which are basically morphisms between categories it's time to introduce morphisms between Functors.
Why do we need this? Well in the last example we ended up with an Endofunctor (we mapped a Category into itself). We need to get rid of one of the
Options. We do this with a morphism that maps the composition of 2 Functors F (F composed with F) to the Functor F.
Lets do a concrete example that involves
List
scala> List(1,2,3) map { x => List(x + 1) }
res0: List[List[Int]] = List(List(2), List(3), List(4))
It happened again, but this time we ended up with a
List[List[Int]]! We now need a Natural Transformation to join those lists together, this Natural Transformation is called flatten in Scala.
scala> List(1,2,3) map { x => List(x + 1) } flatten
res1: List[Int] = List(2, 3, 4)
List comes with a method that does both in one step, it's called flatMap.
scala> List(1,2,3) flatMap { x => List(x + 1) }
res2: List[Int] = List(2, 3, 4)
This enables us to chain functions together in a very convenient way.
scala> Some(1) flatMap { x => Some(2) flatMap { y => Some(x + y) } }
res3: Option[Int] = Some(3)
or a little more concise
scala> Some(1) flatMap { x => Some(2) map { y => x + y } }
res4: Option[Int] = Some(3)
Monads
This is actually what makes up a Monad. It's an Endofunctor with two Natural Transformations called unit and join (which is called
flatten in Scala). They are such a fundamental concept that Scala features a syntactic sugar for them, the for-comprehension. Do NOT confuse this with a mere for-loop, this is a different animal. The last example above can be written in the following way:
scala> for {
| x <- br="" some=""> | y <- br="" some=""> | } yield x + y
res5: Option[Int] = Some(3)
//translates to
scala> Some(1) flatMap { x => Some(2) map { y => x + y } }
res6: Option[Int] = Some(3)
Conclusion
This is an introduction on how to think about monads and how to derive them from simple building blocks. They are a very powerful abstraction that pops up all the time. You use them all the time without knowing. Since programming is all about abstraction it's useful to recognize those underlying patterns. Don't let others fool you into believing that this is just some bizarre functional stuff that has no use in the real world. This is just the tip of the iceberg, an amazing journey lies ahead. Let's learn!
Samstag, 9. Juli 2011
DTrace and it's impact on JVM performance
So, I did this blogpost that gave a very shallow introduction to what DTrace can do with the JVM and I got an amazing feedback after that. Actually I'm quite flattered that it had such an impact.
One message I received pointed out that DTrace probes are having a significant impact on the JVM's performance, he (the author) would never recommend using the DTrace JVM integration, and then he pointed me… well, a product by his own company (which shall remain nameless, since I don't want to advertise commercial products on this blog). To be fair: I admit that I would do the same. When you put a lot of effort into a piece of software you do it for reasons you believe in.
However, he gave me a link to some benchmarks that showed some impressive results and diagrams about how much the performance of the JVM suffers when DTrace probes are enabled. The bar diagrams were scary, DTrace looked pretty bad compared to the "rival" technology (you can't really call it a rival since DTrace has a completely different objective). But something was fishy about this, and that was: the axes of the diagrams were not labeled. They did show a small blue bar and a big green bar and nothing else. The code for the test case was provided as a gif (no copy and paste to reproduce the results nice and easy). Numbers were not put into any perspective. And blog comments were disabled.
None the less, this was interesting enough to start a little bit of research on the topic.
The benchmarks seemed to focus on how much enabled probes did slow down method calls. I personally don't like benchmarks that use extremely unrealistic cases as a foundation (look how fast I can increment them integers in a tight loop suckaz! Mah language iz teh fast!) but this time I will just go with the flow because this is pretty much what they did in that benchmark (don't try this at home kids, use realistic conditions to benchmark your stuff). I'm not using the same test code here but the idea seems to be pretty much the same.
The system I'm running this on is a Thinkpad X201 with a core i7 and 8 gigs of RAM (yeah I know, I'm just compensating, get over it ;)). The operating system is OpenIndiana Build 151-beta with DTrace 1.7. Java is at 1.6.0_25.
The stupid test case I will be using is this Java program:
Once again, this is not a realistic example. I will just use it to demonstrate that there really is an impact.
Wow, this really seems to hurt… the programm is about 35 times slower with DTrace probes enabled.
To put this into perspective I will commit a capital crime in programming. I will compare this stupid program in language A to the same stupid program written in language B. Let language B be Python in this case. To be crystal clear about this: this is NOT a comparison of Python and Java performance. I just need some landmarks to make all those numbers meaningful (at least to some extent since this is a very badly chosen scenario to begin with).
Here is our Python test case.
And the result:
OK, our software runs slower with probes enabled, but we are still faster than Python and Python's performance is acceptable for a lot of use cases. We now have a slower JVM that can be instrumented. So I'd say: No real harm done.
Now let's use those probes and aggregate some real data. For this test I will use a slightly modified version of
Let's run it!
OK, this is A LOT slower, even slower than Python. 4 Minutes!
We are now aggregating data and that means we are copying data from userspace into kernelspace from where it can be fetched by DTrace consumers (in our case the
Now lets use the most naive debugging technique on earth. We will print out "CALLED!" every time our
As we expected, the result is a disaster. Calling print in a tight loop is an extremely stupid thing to do. We could have used a counter that get incremented with every method call, proxy objects, interceptors etc. (all of them would have been significantly faster).
To do something similar like the print example with DTrace I just a another clause to the script:
This addition prints out what happened in 1 second intervals
The performance impact stays pretty much the same with DTrace, we are done in 4 Minutes while we are presented with a readable stream of information.
There are a lot of ways to generate similar data, but most of them require code changes, are not able to do system wide tracing, are limited to one process and/or just one specific runtime.
Conclusion
Tracing the JVM costs (this shows especially in this pathological use case), but DTrace provides us with a very broad spectrum of probes. The JVM ones are just one source of data. We can actually instrument every part of the system with our DTrace script. Maybe a problem is not even related to our program at all, maybe it's NFS misbehaving, something is wrong with the database or there is some heavy IO going on. With DTrace the whole system becomes transparent. This changes the whole "blame game" and that's the whole point of DTrace. Looking at the system as a whole.
The bottom line is: trace the JVM only if you need to and be aware of the performance impact. This tool is for hunting down problems that are very hard or even impossible to analyze with traditional tools. I did use it to trace obscure memory leaks and dead-locks (both in non-Java contexts) and I was able to exactly pinpoint the culprit.
Don't use DTrace when there is a tool that does a better job for this specific task. Use it wisely. Anyway, it's a great utility to have in your toolbox.
Last but not least: use realistic use cases for benchmarking, label your diagram axes, and compare tools that have the same objective.
One message I received pointed out that DTrace probes are having a significant impact on the JVM's performance, he (the author) would never recommend using the DTrace JVM integration, and then he pointed me… well, a product by his own company (which shall remain nameless, since I don't want to advertise commercial products on this blog). To be fair: I admit that I would do the same. When you put a lot of effort into a piece of software you do it for reasons you believe in.
However, he gave me a link to some benchmarks that showed some impressive results and diagrams about how much the performance of the JVM suffers when DTrace probes are enabled. The bar diagrams were scary, DTrace looked pretty bad compared to the "rival" technology (you can't really call it a rival since DTrace has a completely different objective). But something was fishy about this, and that was: the axes of the diagrams were not labeled. They did show a small blue bar and a big green bar and nothing else. The code for the test case was provided as a gif (no copy and paste to reproduce the results nice and easy). Numbers were not put into any perspective. And blog comments were disabled.
None the less, this was interesting enough to start a little bit of research on the topic.
The benchmarks seemed to focus on how much enabled probes did slow down method calls. I personally don't like benchmarks that use extremely unrealistic cases as a foundation (look how fast I can increment them integers in a tight loop suckaz! Mah language iz teh fast!) but this time I will just go with the flow because this is pretty much what they did in that benchmark (don't try this at home kids, use realistic conditions to benchmark your stuff). I'm not using the same test code here but the idea seems to be pretty much the same.
The system I'm running this on is a Thinkpad X201 with a core i7 and 8 gigs of RAM (yeah I know, I'm just compensating, get over it ;)). The operating system is OpenIndiana Build 151-beta with DTrace 1.7. Java is at 1.6.0_25.
The stupid test case I will be using is this Java program:
class Test {
public int callTest(int i) {
if (i != 0)
callTest(i - 1);
return i;
}
public static void main(String[] args) {
Test t = new Test();
long starttime = System.currentTimeMillis();
for (int i = 0; i < 1000000; i++)
callTest(100)
long endtime = System.currentTimeMillis();
System.out.println(endtime - starttime);
}
}
Once again, this is not a realistic example. I will just use it to demonstrate that there really is an impact.
> java Test
118
> java -XX:+ExtendedDTraceProbes Test
4106
Wow, this really seems to hurt… the programm is about 35 times slower with DTrace probes enabled.
To put this into perspective I will commit a capital crime in programming. I will compare this stupid program in language A to the same stupid program written in language B. Let language B be Python in this case. To be crystal clear about this: this is NOT a comparison of Python and Java performance. I just need some landmarks to make all those numbers meaningful (at least to some extent since this is a very badly chosen scenario to begin with).
Here is our Python test case.
import time
def callTest(i):
if not i == 0:
callTest(i - 1)
return i
r = range(0,1000000)
starttime = time.time()
for i in r:
callTest(100)
endtime = time.time()
print (endtime - starttime)
And the result:
> python test.py
21.9892270565
OK, our software runs slower with probes enabled, but we are still faster than Python and Python's performance is acceptable for a lot of use cases. We now have a slower JVM that can be instrumented. So I'd say: No real harm done.
Now let's use those probes and aggregate some real data. For this test I will use a slightly modified version of
j_methodcalls.d, a script by Brendan Gregg that is shipped with the DTrace Toolkit. The script is licensed under CDDL but I did remove the license header here to make it more concise and blog-friendly.
#!/usr/sbin/dtrace -Zs
hotspot$target:::method-entry
{
this->class = (char *)copyin(arg1, arg2 + 1);
this->class[arg2] = '\0';
this->method = (char *)copyin(arg3, arg4 + 1);
this->method[arg4] = '\0';
this->name = strjoin(strjoin(stringof(this->class), "."),
stringof(this->method));
@calls[pid, this->name] = count();
}
Let's run it!
> pfexec ./j_methodcalls.d -c "java -XX:+ExtendedDTraceProbes Test"
[…snip…]
249126
OK, this is A LOT slower, even slower than Python. 4 Minutes!
We are now aggregating data and that means we are copying data from userspace into kernelspace from where it can be fetched by DTrace consumers (in our case the
dtrace command line tool). What data did we get in this amount of time? Actually: A LOT. It flooded my terminal and I had to pipe the result into less to be able to read all of it. DTrace recorded every method call that happened in the JVM while the program was running. It counted the calls per method, and the class the method belongs to. Keep in mind that we did not need to modify our code to get these results. We don't even need to restart a running JVM to enable the probes we can activate them by using the jinfo command. And we could have used DTrace to gather system wide data in the same script (something I might demonstrate sometime on this blog)Now lets use the most naive debugging technique on earth. We will print out "CALLED!" every time our
callTest method gets called (if you ever did this before you know how disastrous the result well be). This gives us pretty much no information. We just know that a particular method has been called and we need to modify our code, recompile and load it into running JVM.
> time java Test
[…snip…]
CALLED!
5958514
real 1h39m18.15s
user 3m53.29s
sys 7m55.68s
As we expected, the result is a disaster. Calling print in a tight loop is an extremely stupid thing to do. We could have used a counter that get incremented with every method call, proxy objects, interceptors etc. (all of them would have been significantly faster).
To do something similar like the print example with DTrace I just a another clause to the script:
tick-1s {
printa(@calls);
trunc(@calls);
}
This addition prints out what happened in 1 second intervals
1 75779 :tick-1s
4028 Test.callTest 400379
1 75779 :tick-1s
4028 Test.callTest 404720
1 75779 :tick-1s
4028 Test.callTest 402135
1 75779 :tick-1s
4028 Test.callTest 398934
253064
dtrace: pid 4028 has exited
real 4m14.23s
user 4m13.89s
sys 0m0.46s
The performance impact stays pretty much the same with DTrace, we are done in 4 Minutes while we are presented with a readable stream of information.
There are a lot of ways to generate similar data, but most of them require code changes, are not able to do system wide tracing, are limited to one process and/or just one specific runtime.
Conclusion
Tracing the JVM costs (this shows especially in this pathological use case), but DTrace provides us with a very broad spectrum of probes. The JVM ones are just one source of data. We can actually instrument every part of the system with our DTrace script. Maybe a problem is not even related to our program at all, maybe it's NFS misbehaving, something is wrong with the database or there is some heavy IO going on. With DTrace the whole system becomes transparent. This changes the whole "blame game" and that's the whole point of DTrace. Looking at the system as a whole.
The bottom line is: trace the JVM only if you need to and be aware of the performance impact. This tool is for hunting down problems that are very hard or even impossible to analyze with traditional tools. I did use it to trace obscure memory leaks and dead-locks (both in non-Java contexts) and I was able to exactly pinpoint the culprit.
Don't use DTrace when there is a tool that does a better job for this specific task. Use it wisely. Anyway, it's a great utility to have in your toolbox.
Last but not least: use realistic use cases for benchmarking, label your diagram axes, and compare tools that have the same objective.
Mittwoch, 6. Juli 2011
DTrace and Scala
DTrace is one of my favorite technologies ever, it absolutely redefined my view on software and how it can be debugged. In a lot of situations you are basically forced to debug your software by vomiting print-statements just all over the place. To make things worse you have to take them out when you are actually shipping your software, leaving you blind to errors to come. In other cases you probably are missing some print-statements to trace down a very nasty bug, or the bug is hidden in some 3rd-party lib that would take hours to recompile with your new print statement (don't start thinking about libs that you don't have the code of…). In most cases this can become a very unpleasant situation (to put it mildly). DTrace takes a lot of the hassle away by providing you with a mechanism that can modify your program while it's running without stopping it.
Pretty amazing, huh? Wait, it gets better. You can even trace into the runtime of some high level languages (if they provide the probes). This is also true for the JVM, and that means we can instrument a running Scala program.
In this post I will walk you through a very basic script that shows what methods from
OK, enough sweet-talk. Time to get our hands dirty by writing a very simple Scala program:
This just prints out "HALLO!" in 1 second intervals (not exactly rocket science, but I put a little sugar on top of it by replacing the while loop with a tail recursive function for fun and profit).
What's that? When running my program DTrace is not showing me ANY probes!!?!?! FFFFUUUUUUUU! That's because we have to enable them first, we can instruct a running JVM to do this by using
The JVM probes are now armed and "dangerous" (just kiddin') and you will have access to the
Now lets write the DTrace script. Keep in mind: this script is running in kernel space, so we have to copy in some information from userspace, we do this by using
This thing will fire whenever a function from
The output will look like this:
WTF IS THIS I DON'T EVEN!?!?!? TL;DR
OK, this is not even the tip of the iceberg but I think I will wrap it up because there is a lot of ground to cover when it comes to DTrace. If you are hungry for more you should check out the "DTrace Review" by @bcantrill, this stuff will blow your mind (seriously, WATCH IT!) or buy the book by @brendangregg. I will make sure to dig deeper on the topic, so stay tuned. Tell me what you think, good tool or best tool? :P
Pretty amazing, huh? Wait, it gets better. You can even trace into the runtime of some high level languages (if they provide the probes). This is also true for the JVM, and that means we can instrument a running Scala program.
In this post I will walk you through a very basic script that shows what methods from
Predef get called by your program. DTrace is a tool that reaches deep down into the lower levels of your machine. Sounds scary? Nah not really, one of the design goals of DTrace is safety. Your scripts run in a managed environment that keeps it from doing harmful things.OK, enough sweet-talk. Time to get our hands dirty by writing a very simple Scala program:
import scala.annotation.tailrec
object Main {
@tailrec
def sayHallo() {
println("HALLO!")
Thread.sleep(1000)
sayHallo()
}
def main(args: Array[String]) {
sayHallo()
}
}
This just prints out "HALLO!" in 1 second intervals (not exactly rocket science, but I put a little sugar on top of it by replacing the while loop with a tail recursive function for fun and profit).
What's that? When running my program DTrace is not showing me ANY probes!!?!?! FFFFUUUUUUUU! That's because we have to enable them first, we can instruct a running JVM to do this by using
jinfo. Since I only got one JVM running on this box I will fetch the PID with pgrep.
jinfo -flag +ExtendedDTraceProbes $(pgrep java)
The JVM probes are now armed and "dangerous" (just kiddin') and you will have access to the
hotspot provider.Now lets write the DTrace script. Keep in mind: this script is running in kernel space, so we have to copy in some information from userspace, we do this by using
copyin and we have to NULL-terminate the strings ourselves. Yep, this is what it feels like to program low-level stuff, it's not as pretty as FP but aaaaaaaaaanyway, here is the little bugger.
#!/usr/sbin/dtrace -s
#pragma D option quiet
hotspot$$target:::method-entry
{
this->class = (char *) copyin(arg1, arg2 + 1);
this->class[arg2] = '\0';
self->tracefunc = stringof(this->class);
}
hotspot$$target:::method-entry
/self->tracefunc == "scala/Predef$"/
{
this->method = (char *) copyin(arg3, arg4 + 1);
this->method[arg4] = '\0';
printf("%s %Y\n", stringof(this->method), walltimestamp);
self->tracefunc = 0;
}
hotspot$$target:::method-entry
/self->tracefunc/
{
self->tracefunc = 0;
}
This thing will fire whenever a function from
Predef is called and will give us the function name (in our test case this is just println) and the time when this function was being called. I run this on OpenIndiana build151-beta by issuing pfexec dtrace ./tracescript.d -p $(pgrep java) after I enabled the hotspot provider on the JVM. (pfexec is kind of like sudo, just use whatever gives you the permission to run dtrace on your box)The output will look like this:
println 2011 Jul 6 21:27:34
println 2011 Jul 6 21:27:35
println 2011 Jul 6 21:27:36
println 2011 Jul 6 21:27:37
println 2011 Jul 6 21:27:38
println 2011 Jul 6 21:27:39
println 2011 Jul 6 21:27:40
println 2011 Jul 6 21:27:41
println 2011 Jul 6 21:27:42
println 2011 Jul 6 21:27:43
println 2011 Jul 6 21:27:44
println 2011 Jul 6 21:27:45
println 2011 Jul 6 21:27:46
println 2011 Jul 6 21:27:47
println 2011 Jul 6 21:27:48
println 2011 Jul 6 21:27:49
println 2011 Jul 6 21:27:50
println 2011 Jul 6 21:27:51
println 2011 Jul 6 21:27:52
println 2011 Jul 6 21:27:53
println 2011 Jul 6 21:27:54
println 2011 Jul 6 21:27:55
WTF IS THIS I DON'T EVEN!?!?!? TL;DR
OK, this is not even the tip of the iceberg but I think I will wrap it up because there is a lot of ground to cover when it comes to DTrace. If you are hungry for more you should check out the "DTrace Review" by @bcantrill, this stuff will blow your mind (seriously, WATCH IT!) or buy the book by @brendangregg. I will make sure to dig deeper on the topic, so stay tuned. Tell me what you think, good tool or best tool? :P
English, please!
It took me quite some time to make up my mind about this… should I switch the language of this blog to English or should leave it just the way it is. I think it's worth to give this a try, since twitter brought me towards a mostly non-German audience which I also would like to address. This is also a great opportunity to polish my language skills a little since they got kind of neglected in the last few years (so bear with me ;)
So, to all my new readers, here is a little overview on what this blog is about. I work as a developer at a German software company so quite a lot of stuff you will find here will focus on programming and debugging. In my free time I like to play around with different operating systems (mainly Illumos, FreeBSD and DragonFlyBSD) so this place will be stuffed with information about OS specific wizardry :3
OK, folks! Let's give this a shot!
So, to all my new readers, here is a little overview on what this blog is about. I work as a developer at a German software company so quite a lot of stuff you will find here will focus on programming and debugging. In my free time I like to play around with different operating systems (mainly Illumos, FreeBSD and DragonFlyBSD) so this place will be stuffed with information about OS specific wizardry :3
OK, folks! Let's give this a shot!
Mittwoch, 23. Februar 2011
Dynamic-Dispatch in Scala
Ich bin ja hin und wieder schon mal richtig neidisch auf Clojures Multimethods, möchte aber nicht wirklich meine statisch typisierte Welt verlassen (auch wenn Clojure vermutlich die einzige Sprache wäre die ich von allen dynamischen Sprachen wählen würde).
Lange Rede, kurzer Unsinn: Ich will Multimethods in Scala! Auf den ersten Blick klingt das als würde man sich eine Sprache zu etwas hinbiegen wollen was sie gar nicht ist, aber in Scala geht das ganze erschreckend schmerzlos.
Zum dispatchen verwende ich einfach ein paar partielle Funktionen die man mit
In Aktion sieht das Ganze so aus:
Wie man schön sehen kann werden auch die nicht abgedeckten Fälle behandelt indem wir einfach eine
Viel Spass beim ausprobieren ;)
Hier ein kleiner Nachtrag wozu Multimethods gut sind. Für alle die Das Visitorpattern schon immer gehasst haben ;)
Lange Rede, kurzer Unsinn: Ich will Multimethods in Scala! Auf den ersten Blick klingt das als würde man sich eine Sprache zu etwas hinbiegen wollen was sie gar nicht ist, aber in Scala geht das ganze erschreckend schmerzlos.
Zum dispatchen verwende ich einfach ein paar partielle Funktionen die man mit
orElse beliebig aneinanderketten kann. Der Vorteil davon ist das ich keinen Code aufmachen muss wenn ich einen weiteren Fall hinzufügen will sondern durch Komposition zum Ziel komme.
trait Dynamic
class A extends Dynamic
class B extends Dynamic
object Dynamic {
// 2 dynamische argumente
type Dyn2 = (Dynamic, Dynamic)
// infix notation fuer PartialFunction
type ~>[A, B] = PartialFunction[A, B]
val f: Dyn2 ~> String = {
case (a: A, b: A) => "A A"
}
val g: Dyn2 ~> String = {
case (a: A, b: B) => "A B"
}
// d ist die dispatchfunktion
def dispatch[A, B](d: A ~> B, a: A): Option[B] =
if (d.isDefinedAt(a)) Some(d(a))
else None
}
In Aktion sieht das Ganze so aus:
scala> :l ./dynamic.scala
Loading ./dynamic.scala...
defined trait Dynamic
defined class A
defined class B
defined module Dynamic
scala> import Dynamic._
import Dynamic._
scala> dispatch(f, (new A, new A))
res0: Option[String] = Some(A A)
scala> dispatch(f, (new A, new B))
res1: Option[String] = None
scala> dispatch(f orElse g, (new A, new B))
res2: Option[String] = Some(A B)
Wie man schön sehen kann werden auch die nicht abgedeckten Fälle behandelt indem wir einfach eine
Option verwenden, dadurch könnten wir unseren Dispatch auch in einer for-Comprehension verwenden (Monaden für den Gewinn!)Viel Spass beim ausprobieren ;)
Hier ein kleiner Nachtrag wozu Multimethods gut sind. Für alle die Das Visitorpattern schon immer gehasst haben ;)
Montag, 21. Februar 2011
Scala: Besseres Pattern-Matching mit Sealed Classes
Neulich bin ich auf eine recht interessante Mail in der jvm-languages Google Group gestossen. Ein Vergleich von Scala und OCaml! Wie spannend!
Besonders interessant fand ich folgendes Beispiel:
OCaml liefert hier beim kompilieren einen Fehler: Das Patternmatching würde nicht alle Möglichkeiten erschöpfen. Zusätzlich bekommt man noch ein Beispiel wo der Match fehlschlägt. Tolle Sache! Das Scala Beispiel sieht hier doch eher ernüchternd aus.
Anscheinend ist Scala wohl nicht in der Lage herauszufinden welche Fälle möglich sind und kann so nicht testen ob das Matching auch wirklich alle Möglichkeiten ausschöpft. Aber genau dieser Effekt lässt sich mit Sealed Classes erreichen.
Als Beispiel definiere ich einen eigenen Booltypen mit einem Subtypen für jeweils true und false (ich nehme hier Objects da wir nicht mehr als eine Instanz von True bzw. False brauchen). Das gleiche Beispiel schlägt hier mit einer ähnlichen Fehlermeldung wie bei OCaml fehl:
Die Sealed Class hat zur Folge das wir keine weiteren Subtypen außerhalb unser Quelldatei anlegen können, der Compiler hat jetzt also alle Information die er braucht um den Match zu prüfen.
Mehr zum diesem Thema gibt es wieder mal bei Mario Gleichmann.
Besonders interessant fand ich folgendes Beispiel:
# let foo = function
| true, _ -> 0
| _, false -> 1;;
Warning P: this pattern-matching is not exhaustive.
Here is an example of a value that is not matched:
(false, true)
val foo : bool * bool -> int =
OCaml liefert hier beim kompilieren einen Fehler: Das Patternmatching würde nicht alle Möglichkeiten erschöpfen. Zusätzlich bekommt man noch ein Beispiel wo der Match fehlschlägt. Tolle Sache! Das Scala Beispiel sieht hier doch eher ernüchternd aus.
scala> def foo(b: (Boolean, Boolean)): Int = b match {
| case (true, _) => 0
| case (_, false) => 0
| }
foo: ((Boolean, Boolean))Int
scala> foo(false, true)
scala.MatchError: (false,true)
at .foo(:4)
at .(:5)
at .()
at java.lang.Class.initializeClass(libgcj.so.81)
at RequestResult$.(:3)
at RequestResult$.()
at java.lang.Class.initializeClass(libgcj.so.81)
at RequestResult$result()
at java.lang.reflect.Method.invoke(libgcj.so.81)
...
Anscheinend ist Scala wohl nicht in der Lage herauszufinden welche Fälle möglich sind und kann so nicht testen ob das Matching auch wirklich alle Möglichkeiten ausschöpft. Aber genau dieser Effekt lässt sich mit Sealed Classes erreichen.
Als Beispiel definiere ich einen eigenen Booltypen mit einem Subtypen für jeweils true und false (ich nehme hier Objects da wir nicht mehr als eine Instanz von True bzw. False brauchen). Das gleiche Beispiel schlägt hier mit einer ähnlichen Fehlermeldung wie bei OCaml fehl:
abstract sealed class MyBool
case object True extends MyBool
case object False extends MyBool
object MyBool {
def test(a: (MyBool, MyBool)): Unit = a match {
case (True, _) => ()
case (_, False) => ()
}
}
MyBool.scala:6: warning: match is not exhaustive!
missing combination False True
def test(a: (MyBool, MyBool)): Unit = a match {
^
one warning found
Die Sealed Class hat zur Folge das wir keine weiteren Subtypen außerhalb unser Quelldatei anlegen können, der Compiler hat jetzt also alle Information die er braucht um den Match zu prüfen.
Mehr zum diesem Thema gibt es wieder mal bei Mario Gleichmann.
Montag, 31. Januar 2011
Scala: Spass mit Streams
Ich wurde ja neulich mal gefragt ob ich mal was über Streams in Scala bloggen könnte (auch wenn es hier eher um IO ging). Da mir da allerdings ein gutes Beispiel gefehlt hat, habe ich das erst einmal auf die lange Bank geschoben (irgendwie blogge ich derzeit glaub ich etwas zu wenig :/).
Aber ok, dank eines hervorragenden Posts von Mario Gleichmann bin ich dann doch noch auf eine ganz brauchbare Idee gekommen. (Schon allein Vollständigkeit halber empfehle ich den Post von Mario zu lesen!)
Im folgenden Code geht es einfach nur darum die nachfolgende Jahreszeit zu bestimmen.
Um die Jahreszeiten abzubilden nutze ich hier einen Stream (also eine unendliche Liste) die ich rekursiv durchgehe bis ich die aktuelle Jahreszeit gefunden habe und einfach die Nächste ausgebe.
Dieses Beispiel wirkt zugegebenermaßen etwas wie mit Kanonen auf Spatzen zu schiessen, zeigt aber die Ausdrucksstärke von Streams und Lazy-Evaluation. Ein sehr realer Anwendungsfall ist hier z.B. ein Roundrobin-Verfahren. Anstelle also stumpf eine Liste durchzugehen und am Ende wieder an den Anfang springen in Zukunft einfach mal über einen Stream nachdenken ;)
Aber ok, dank eines hervorragenden Posts von Mario Gleichmann bin ich dann doch noch auf eine ganz brauchbare Idee gekommen. (Schon allein Vollständigkeit halber empfehle ich den Post von Mario zu lesen!)
Im folgenden Code geht es einfach nur darum die nachfolgende Jahreszeit zu bestimmen.
sealed abstract class Season
case object Spring extends Season
case object Summer extends Season
case object Fall extends Season
case object Winter extends Season
object Main {
lazy val seasons: Stream[Season] =
Spring #:: Summer #:: Fall #:: Winter #:: seasons
def nextSeason(now: Season): Season = {
@scala.annotation.tailrec
def getNextFromStream(s: Stream[Season]): Season =
s match {
case a @ x #:: y #:: _ =>
if (now eq x)
y
else
getNextFromStream(a tail)
}
getNextFromStream(seasons)
}
def main(args: Array[String]): Unit = {
println(nextSeason(Spring))
println(nextSeason(Summer))
println(nextSeason(Fall))
println(nextSeason(Winter))
}
}
Um die Jahreszeiten abzubilden nutze ich hier einen Stream (also eine unendliche Liste) die ich rekursiv durchgehe bis ich die aktuelle Jahreszeit gefunden habe und einfach die Nächste ausgebe.
Dieses Beispiel wirkt zugegebenermaßen etwas wie mit Kanonen auf Spatzen zu schiessen, zeigt aber die Ausdrucksstärke von Streams und Lazy-Evaluation. Ein sehr realer Anwendungsfall ist hier z.B. ein Roundrobin-Verfahren. Anstelle also stumpf eine Liste durchzugehen und am Ende wieder an den Anfang springen in Zukunft einfach mal über einen Stream nachdenken ;)
Samstag, 25. Dezember 2010
Die Sache mit Apple
"Hallo, ich bin raichoo und ich habe ein MacBook". Das sind Sätze mit denen man in vielen Kreisen schnell Freunde finden kann (Vorsicht Ironie!). Aber mal ehrlich, in den letzten Tagen kann man sich besonders schöne Kommentare gefallen lassen. Mein Favorit ist hier "Apfelnazi" (Goodwin's Law lässt grüßen). Die Diskussionskultur lässt hier ziemlich zu wünschen übrig, wer Leute nach dem Betriebssystem beurteilt hat in meinen Augen doch eher einen massiven Schaden.
Aber was bewegt die Leute dazu so zu reagieren? Auf der einen Seite mag das ganze auf einem pseudoelitären Gefühl basieren. Apple ist Mainstream. Gegen Mainstream sein ist rebellisch, cool und was weiß ich noch alles. Ich bin ja selber keine Ausnahme, meine Lebensweise ist sicherlich nicht die eines Ottonormalverbrauchers. Aber diese extremen Anfeindungen gehen doch echt schon zu weit.
Doch zurück zu den Beweggründen. Wikileaks ist im Moment sicherlich ein sehr bewegendes Thema, und das Verhalten vieler Firmen wie Visa, Paypal, Mastercard und auch Apple ist nicht nur verwerflich, es ist in meinen Augen auch undemokratisch.
Was also machen? Boykottieren? Guter Plan, Boykott war immer schon die wirksamste Waffe des Verbrauchers, denkt man sich. Die Problematik die sich hieraus aber ergibt ist folgende: der überwiegende Großteil der Firmen wird so reagieren. Ein bisschen Druck vom Staat eventuell noch ein paar Zuwendungen unter der Hand und schon wird der Stecker gezogen. Unsere Freiheit wird also potentiell von jeder Firma beschnitten die nur den nötigen Einfluss hat, oben genannte Beispiele befinden sich nur gerade in dieser Situation besonders in so einer Position. Man kann sich also praktisch totboykottieren, der Hydra wachsen die Köpfe einfach nach.
Wie also reagieren? Alle Apple Produkte aus dem Fenster werfen in den nächsten Mediamarkt rennen, ein Plastikwaffeleisen kaufen das nach einem Jahr auseinanderfällt und Ubuntu installieren? Ich sag es ehrlich: Linux ist derzeit einfach technisch keine Alternative für mich, und FreeBSD's Treibersituation auf Notebooks ist auch nicht wirklich berauschend. Windows mag ich nicht, usw.. Erreicht wird damit doch praktisch eh nichts. In meinen Augen ist beim Thema Wikileaks das effektivste Mittel doch eh: so hartnäckig spenden wie es nur geht. Wege dafür gibt es genug. Unterstützung ist weitaus mächtiger als Boykott.
Wenn mir also jemand ein Notebook mit 13" Formfaktor, 6 Stunden+ Akkulaufzeit, Alugehäuse, großen Touchpad auf dem mindestens ein FreeBSD rennt (damit meine ich das ALLE Komponenten auch zuverlässige Treiber haben) und das dazu auch noch KEINERLEI Komponenten von irgendeiner Firma enthält die irgendwo mal Scheisse gebaut hat zeigen kann, bring it on. (Das ist ernst gemeint, würde mich interessieren ob es da draußen irgendwas gibt was dem auch nur ansatzweise entspricht)
Aber was bewegt die Leute dazu so zu reagieren? Auf der einen Seite mag das ganze auf einem pseudoelitären Gefühl basieren. Apple ist Mainstream. Gegen Mainstream sein ist rebellisch, cool und was weiß ich noch alles. Ich bin ja selber keine Ausnahme, meine Lebensweise ist sicherlich nicht die eines Ottonormalverbrauchers. Aber diese extremen Anfeindungen gehen doch echt schon zu weit.
Doch zurück zu den Beweggründen. Wikileaks ist im Moment sicherlich ein sehr bewegendes Thema, und das Verhalten vieler Firmen wie Visa, Paypal, Mastercard und auch Apple ist nicht nur verwerflich, es ist in meinen Augen auch undemokratisch.
Was also machen? Boykottieren? Guter Plan, Boykott war immer schon die wirksamste Waffe des Verbrauchers, denkt man sich. Die Problematik die sich hieraus aber ergibt ist folgende: der überwiegende Großteil der Firmen wird so reagieren. Ein bisschen Druck vom Staat eventuell noch ein paar Zuwendungen unter der Hand und schon wird der Stecker gezogen. Unsere Freiheit wird also potentiell von jeder Firma beschnitten die nur den nötigen Einfluss hat, oben genannte Beispiele befinden sich nur gerade in dieser Situation besonders in so einer Position. Man kann sich also praktisch totboykottieren, der Hydra wachsen die Köpfe einfach nach.
Wie also reagieren? Alle Apple Produkte aus dem Fenster werfen in den nächsten Mediamarkt rennen, ein Plastikwaffeleisen kaufen das nach einem Jahr auseinanderfällt und Ubuntu installieren? Ich sag es ehrlich: Linux ist derzeit einfach technisch keine Alternative für mich, und FreeBSD's Treibersituation auf Notebooks ist auch nicht wirklich berauschend. Windows mag ich nicht, usw.. Erreicht wird damit doch praktisch eh nichts. In meinen Augen ist beim Thema Wikileaks das effektivste Mittel doch eh: so hartnäckig spenden wie es nur geht. Wege dafür gibt es genug. Unterstützung ist weitaus mächtiger als Boykott.
Wenn mir also jemand ein Notebook mit 13" Formfaktor, 6 Stunden+ Akkulaufzeit, Alugehäuse, großen Touchpad auf dem mindestens ein FreeBSD rennt (damit meine ich das ALLE Komponenten auch zuverlässige Treiber haben) und das dazu auch noch KEINERLEI Komponenten von irgendeiner Firma enthält die irgendwo mal Scheisse gebaut hat zeigen kann, bring it on. (Das ist ernst gemeint, würde mich interessieren ob es da draußen irgendwas gibt was dem auch nur ansatzweise entspricht)
Montag, 20. September 2010
Scala: Klassen erweitern mal anders
Datentypen erweitern ist mehr oder weniger etwas das man des öfteren machen muss, und das ist nicht umbedingt immer unproblematisch. Nehmen wir mal an wir hätten folgende Klasse:
Ganz klar: hochkomplizierter Code.
wir Objekte dieser Klasse in unserem Code nahtlos verwenden können. Diese Methode nennen wir
Jetzt gibt es verschiedene Wege wie wir diese Methode in
Eine Lösung ist hier ziemlich elegant mit Traits, Selftyping und Structural Types zu machen.
Als erstes definieren wir einen Structural Type der eine genaue Schnittstelle definiert. Wir nennen ihn
Was wir jetzt brauchen ist unsere generalisierte Implementation von
Wie man sieht muss der muss der Typparameter
Alles was wir jetzt noch machen müssen ist das Trait
Das ganze Zeit mal wieder wie extrem mächtig das Scala Typsystem ist und was für elegante Features die Sprache zum lösen immer wiederkehrender Probleme bietet. Ich hoffe dieses Beispiel gibt euch eine kleine Idee was man damit machen kann. ;)
class Test(val x: Int)
Ganz klar: hochkomplizierter Code.
Test quillt an Features nur so über, aber uns fehlt gerade eine Methode damitwir Objekte dieser Klasse in unserem Code nahtlos verwenden können. Diese Methode nennen wir
bla. Das Problem ist allerdings: Wir haben den Code von Test nicht und können ihn deswegen nicht um bla erweitern.Jetzt gibt es verschiedene Wege wie wir diese Methode in
Test einbinden könnten. Der offensichtlichste ist sicherlich Vererbung. Blöd ist nur: Wir haben bla schon geschrieben und würden ihn ungern duplizieren, vor allem weil wir den Code bereits schön generalisiert haben.Eine Lösung ist hier ziemlich elegant mit Traits, Selftyping und Structural Types zu machen.
Als erstes definieren wir einen Structural Type der eine genaue Schnittstelle definiert. Wir nennen ihn
xAble und er macht nichts weiter als zu definieren das alle Klassen die eine Methode x anbieten vom Typ xAble sind.
type xAble = { def x: Int }
Was wir jetzt brauchen ist unsere generalisierte Implementation von
bla. Hier verwenden wir, Selftyping. Das ist ein Feature das es uns erlaubt innerhalb eines Traits so zu tun als wäre es von einem bestimmten Typ ohne davon zu erben.
trait Gimme[A <: xAble] {
self: A =>
def bla(): Unit = println(self.x + 1)
}
Wie man sieht muss der muss der Typparameter
A ein Subtyp von xAble sein, also die methode x liefern.Alles was wir jetzt noch machen müssen ist das Trait
Gimme in das Objekt des Typs Test zu mischen und wir haben das ganze um bla erweitert.
object Main {
class Test(val x: Int)
type xAble = { def x: Int }
trait Gimme[A <: xAble] {
self: A =>
def bla(): Unit = println(self.x + 1)
}
def main(args: Array[String]): Unit = {
val t = new Test(666) with Gimme[Test]
t.bla()
}
}
Das ganze Zeit mal wieder wie extrem mächtig das Scala Typsystem ist und was für elegante Features die Sprache zum lösen immer wiederkehrender Probleme bietet. Ich hoffe dieses Beispiel gibt euch eine kleine Idee was man damit machen kann. ;)
Donnerstag, 24. Juni 2010
FreeBSD 8.1 und die JVM
Alios hat gerade seinen Server auf den neusten Stand gebracht und prompt funktionierte Sakura natürlich wieder nicht. Das Problem war das die JVM wohl anscheinen den IPv6 Stack genommen hat und sich damit nicht verträgt, ein Workaround ist die JVM einfach dazu zu zwingen den IPv4 Stack zu nehmen. Das geht mit der
Wenn euch also das nächste mal ein Socket beim Connect aus unerfindlichen Gründen abraucht, versucht es mal damit.
-Djava.net.preferIPv4Stack=true Option.Wenn euch also das nächste mal ein Socket beim Connect aus unerfindlichen Gründen abraucht, versucht es mal damit.
Abonnieren
Posts (Atom)