Workspace upgrade

Unfortunately my old 24″ Samsung monitor died yesterday. I have been thinking about upgrading my monitor setup for some time now, so this was the final push I needed. I bought a 29″ ultra-wide monitor (LG 29UM58-P). So far so good, I’m still getting used all the horizontal space and the vibrant colors. I’m keeping my other 24″ for now, but I might end up buying two smaller monitors to put on both sides of the 29″er.
Edit: Having the IDE set up like this feels just right
(Project structure|Code editor|Console output/debugger):
screenshot_20161101_131922

Advertisements

Trust me, I know better

Today I was working on fixing some unstable tests. We use Gradle as our build tool, and we’ve built so much on top of the regular Scala-Java-Javascript plugins that it’s staggering: our Gradle files contain more than 50K lines. We added abstractions, IDE management tools, complex test and binary generation with application outputs and startup script generation. There’s probably much-much more I don’t actually know about as I’m not a build infra developer. Most of these features try to hide and abstract away the complexities of managing the largest Scala project in the world (and all its little side modules). While I was working on fixing these tests, I wanted to simply print a value to the console to check if the test infra picks up a runtime dependency correctly – both on developer desktops and on CI servers. I wish it was as simple as that though! Gradle itself redirects test output and on top this we also forcibly change the logging settings so everything just works. Except when you want to do something out of the ordinary. I needed to start googling for vanilla Gradle solutions to find out how we actually do it by digging around in our code. It took me good 20 minutes to figure out this seemingly simple thing. It turned out all these were hidden away in some test configs and dependent properties. I managed to achieve what I wanted in the end.

When similar things happen, I really miss simpler days. I feel that most of the time we, developers try to fight a war of I know better. Most of us think that we’ve covered all use cases, all the little configurations are straightforward, the documentation lists all possibilities and if not, well, it can be figured out as it’s all logical. I can see the same thing happening with how the new way of JavaScript development happens in the world: you should use a couple of tools and libs that hide almost everything from you: dependency management, language and browser differences; even language features! All this so we only need to care for what’s important. This is really nice from the outside, but as soon as you need to walk off the regular path, you find yourself in a nightmare of multiple levels of abstractions, hidden little tricks and things that only a handful of people know about. I also understand that this complexity needs to live somewhere, but I feel we’ve gone a bit too far. The learning curve has become steeper than it was ever before.

Unfortunately I can’t offer a solution at this point – not even hypothetical one. Sometimes I just simply start daydreaming of good old days when everything was simple. Or at least simpler.

Add people to email chains, because that’s going to solve the problem!

There has been a systematic issue with one of our CI jobs that runs automated Ui tests. I have sent out an email to all users who hit this issue when it was triggered for their commits, stating that we know of the problem and it’s being investigated and it’s highly unlikely that the problem lies with the failed tests. One of the people I’ve sent this email to actually added the owner of those tests because he didn’t see how his changes could have affected them. I had to explain once again that the issue is systematic and it’s currently being looked at. I also started a discussion with the team managing CI jobs so we could look at the problem together. After some time, the new guy replied to the original thread, and of course added some other people because they should be also involved. This was the third time that I had to explain what was going on, and that we were working on the issue. I also attached the other thread I had with the build team, just in case.

I feel that there’s this fallacy that talking about problems will get them resolved. Talking is more effective if more people do it, so it only makes sense to broadcast everything, even though there’s no chance others can help actually fixing the issue or even if all possible actions have been taken towards finding a resolution. These then cost people working on the actual fixes time and effort, hindering getting it done. But since all these people felt involved, they also think they were part of the fix too. It’s an ego and visibility boost.

The very same thing happens when a client’s request is prioritized and actively looked at, but they insists of having daily catch ups, because sure, that’s going to speed things up! There are hard problems that take time and concentration; continuously interrupting people doing creative work never have helped in human history. What I don’t actually understand is why people doing creative work still do this to others. It’s like they think rules/suggestions/recommendations only apply to others, never to them!

Why getOrElse with map is possibly worse than fold with Option[T]s

Option[T] is probably everyone’s favorite monad in Scala. It’s basically the same as Maybe in Haskell. It provides a nice functional interface with nullable types on the JVM. It also supports type information to help the compiler help us catch errors before runtime. It supports all container-type methods, including foreach, map, getOrElse, head, etc. To transform the value inside the Option, I used to do the following:

scala> val o: Option[Int] = Some(3)
o: Option[Int] = Some(3)

scala> o.map { _ * 2 }
res1: Option[Int] = Some(6)

As you can see, map actually returns a new Option which totally makes sense, as it can’t know beforehand that there’s actually a value:

scala> val x: Option[Int] = None
x: Option[Int] = None

scala> x.map{ _ * 2 }
res3: Option[Int] = None

The usual response to that would be using a getOrElse on the new Option returned by map:

scala> o.map { _ * 2 }.getOrElse(0)
res5: Int = 6

scala> x.map{ _ * 2 }.getOrElse(0)
res6: Int = 0

The only problem with this approach is that because getOrElse needs to be generic enough, we can actually return any type from it. Here’s the signature of the method: def getOrElse[B >: A](default: => B): B, and this lets us do this kind of shenanigans:

scala> x.map{ _ * 2 }.getOrElse("no")
res7: Any = no

Combining this with type deduction, we can actually write code that long lost type information, and would result in runtime errors.

Now comes fold to the rescue! With the following signature def fold[B](ifEmpty: => B)(f: (A) => B): B it can actually achieve the same as map+getOrElse:

scala> o.fold(0){ _ * 2 }
res8: Int = 6

scala> x.fold(0){ _ * 2 }
res9: Int = 0

But saves us from doing possibly stupid things:

scala> x.fold("no"){ _ * 2 }
:13: error: type mismatch;
 found   : Int
 required: String
       x.fold("no"){ _ * 2 }
                       ^

since it requires that both the mapping and the default value have the same type. Plus, it’s more concise and it’s only one method invocation.

Yes, there are cases when we actually want to change the type. For those cases getOrElse can be used; but most of the time using fold is just much safer.

The misconception of getting ready

I have noticed an interesting phenomenon with several people I know and based on some articles I’ve read, it seems it’s more widespread than my social circle. I’ll call it the misconception of getting ready or how to procrastinate actively.

If there is a non-trivial task to finish or a goal to achieve that takes a lot of effort and constant dedication, some people spend so much time on getting ready for it that the actual, main thing seemingly gets pushed in the background. A good example would be a personal software project that needs to be done fairly quickly, but the person spending long times on setting up the environment, getting version control ready, installing a new IDE, playing around with settings or reading a lot on these subjects (and not on the actual project!). Another one I read about a lot is starting some sport or physical training – this usually happens with women according to my experience. They’ll go shopping, buy the most fancy training clothes, water bottles, accessories and who knows what else and they keep pushing back on actually starting. They read a gazillion blog posts and have fancy, nicely set up photos and post them on Instagram without even breaking a sweat.

This is not about proper planning, mind you. I’m all for that. I like to design and plan my software, I like to know where things lead and to decrease the possibility of unpleasant surprises to the minimum. This is something entirely different.

I think the psychological explanation is pretty simple. These people don’t actually want to do the main activities, but they know they should, so instead of procrastinating the good old way (i.e. not doing it at all), they do it actively. This calms them to some extent – they seem to be slowly approaching the end goal without even starting what they despise. It works wonderfully. I have another example that fits this thinking nicely in my opinion: commuting by car. You sit in traffic and spend a lot of time making the car go in the proper direction. You don’t actually do anything really useful, but since it requires you to pay attention, it seems you’re not wasting time. The alternatives, on the other hand offer better options: on public transport you could be reading a book or if you biked, you could get some nice cardio in.

I’m not saying we don’t need to spend time on getting ready; what I find harmful for achieving goals is doing it excessively.

My favorite Aeropress method

I use an Aeropress as one solution to fulfill my caffeine addiction. Here’s my preferred brewing method:

  • Grind 12g of coffee beans
  • Heat water to ~80 degrees
  • Put the Aeropress in the inverted position
  • Put in the ground coffee
  • Pour 50g water
  • Stir it a couple of times (~5)
  • Wait 30 seconds – this should give you enough time to water the filter
  • Pour 150g water
  • Wait 30 seconds without stirring or moving – this should give you enough time screw the filter on
  • Press the Aeropress still in the inverted position slightly, until you can see bubbles seeping through the filter
  • Flip and put the device over your mug – it should be in normal position
  • Wait 2 and a half minutes
  • Use the forearm technique to apply pressure

Depending on the beans, of course, but this should give you a nice, smooth taste with a strong enough kick.

Incremental compilation

I have a strong feeling that we’re doing this whole incremental compilation thing wrong. I have hands-on experience with the Scala compiler (which is known to be one of the badly written ones), but I think this could be applied to probably all languages.

The starting observation is pretty simple: we compile code much more frequently in our IDEs while we’re developing (i.e. continuously modifying the code) than have a fixed point of the code base to generate binaries. And yet, most compilers known to me are designed and written for the latter, and we later tried to force and trick them to work for the first case. There’s a Hungarian proverb that matches the situation: sitting on the horse backwards. I know, I know. This originates in the old days when we didn’t have IDEs and really only the monolithic compilation existed. A lot has changed since most people write their code in vi and compile it separately though. It’s also known that writing compilers is really complex and hard. Why don’t we make it potentially more complex then? :) Let me pitch you the idea.

The current way of things is that we have the compiler that can be invoked on a bunch of source files. It parses those files, builds ASTs and runs complicated algorithms to check if the text has proper syntax and is semantically correct – whatever that means in the language. After this happens, it generates lower-level representation of the text that can be interpreted (in the meaning of understood, not in the CS ‘interpreted’) by either a virtual machine or a physical one. While a developer is using an IDE (Integrated Development Environment) they modify the text in one or more of those files. When the incremental compilation is started -e.g. the developer started a test- the compiler tries to determine the changes and how it affects the dependencies of the modified text. This information is used to invoke the compiler on the minimum number of entities that must be recompiled (entities here can refer to source files, classes or anything that makes sense in the given language). In the case of languages that have a high level of expressiveness -including Scala-, determining the effects and the dependent entities is really complicated. To be on the safe side we usually end up recompiling entities that might not necessarily have to be recompiled. We also need to keep track of the mapping between the original text and the compiler’s internal representation so we can point out errors we have found. IDEs tend to support other useful features such as syntax highlighting, code navigation and code completion. They need to have some idea of what the text actually means to support these features, so they usually have some kind of representation of the text in an object model. This representation can be really close to what the compiler uses – in some cases they don’t need all the information though, letting them dismiss some minor details.

Instead of going through all these hurdles to support the developer write the code, we should let them edit the AST directly. By this I don’t mean having a WYSIWYG graphical editor to build the AST, but all modifications of the text should be mapped to direct modifications of the AST. Since we need to have the representation of the code in the memory and we need to keep the mapping between the text and the compiler’s internal data structures we should have everything we need to achieve this. We could potentially support operations in the compiler that can be easily mapped to the possible textual modifications. Let me give you a few examples in Scala.
Every project started with an empty source file. When the first package is created, that can be fed to the compiler as a package created command. The same applies to classes, methods, variables and basically everything else when they are first created.
When the developer modifies the type of a parameter to a method, we know we need to rerun the typer and check all uses of the method. If the modification happens in the in-memory AST, we should be able to easily trickle down the effects and regenerate only the sub-trees of the AST that we need. We shouldn’t need to try to figure out these on differences between the text if we modified the AST directly.
There are some types of changes that wouldn’t “redraw” the representation, e.g. moving things around in the file or adding a comment would only need to update the position information.
I believe mapping all possible text modifications to compiler commands shouldn’t be too hard; I haven’t gone through all of them though. It would require major compiler redesign, of course.

This approach could also help implementing the aforementioned IDE features: it would give us immediate error reporting for free, update the data for code completion and syntax highlighting, plus it could potentially support more robust refactoring tools.