Maven version ranges inception

Have you ever encountered a situation where your changes to a configuration file (like owasp-suppressions.xml) aren't being reflected? If you're working on a Maven project that uses version ranges, you might be in for a surprise.

In a maven-based project that uses version ranges, there is an implicit expectation may lead to great frustration for developers. Frustration similar to an intermittent issue that is hard to figure out. Let's say you've defined the version range [1,2) for a dependency you also develop. You'd expect this to always point to the latest SNAPSHOT, as long as it is before the release version 2.0.0. However, this expectation may unexpectedly defy expectations, causing the latest SNAPSHOT to not be reflected during a local build.

When you define the version range [1,2)

for a dependency that you also develop for, normally you expect it to behave such that it will always point to the latest SNAPSHOT as long as it is before the release version 2.0.0.

In my case, the dependency was released a few days prior. But due to unrelated build issues, like Docker base Debian images suddenly failing to run apt-update, the release had to be done twice. So instead of release 1.5, the version was bumped to 1.6. Our build process uses the Maven release plugin to automatically update the version to the next development SNAPSHOT. However, due to the build issues, the release for 1.6 had to be done manually, and updating to the correct version of 1.7-SNAPSHOT was forgotten. The dependency project was left at version 1.6-SNAPSHOT.

During a Maven build, the version range resolves to the latest version. In my case, the latest was the release version 1.6, as release versions are considered greater than SNAPSHOTs. Additionally, Maven typically checks both local and remote repositories for a given dependency. So, when the remote repository has the released 1.6, but locally you're still on 1.6-SNAPSHOT, your build will use the release version instead of your local SNAPSHOT version. This causes a situation where your local code changes aren't reflected when you run your code.

I was able to diagnose this problem by looking at the metadata files that maven keep in:

cat ~/.m2/repository/path-to-your-dependency/maven-metadata-<remote-hostname>.xml

Reflection of thoughts #1 - Logs and Multithreaded applications

Ever since I have started working, applications have had to run an increasing number of threads. This is mostly due to the adoption of web applications in favor of desktop applications. Due to this change; One the more difficult work that has come up is looking at the logs and figuring out what happened.

For a significantly busy server, looking at the logs is not as straightforward as I would expect.

1. In the logs, each http request is represented as its own thread, so you can identify which logs belongs to which http request since the thread name is included in the log. However, it would still be hard to hard to follow because multiple threads be adding new lines to the log in a mixed fashion because these threads are running simultaneously.
2. Much more difficult is when each http request would also spawn new background threads where each has their own name which you cannot reliably associate with the parent http request. For example a spawn background thread called acme_Quartz_Worker-9 . With the logs being appended by many threads on-demand.. I think it is impossible to identify which worker thread belong to which http request thread.

There are basic I have learned over the years that are must-have things to keep the logs as helpful as possible:

1. In Java, always make sure to always include the exception when you call logger.error(). Failing to include it may hide valuable clue since most exceptions have a root cause exception that would usually be the most important clue.
2. If available, always add contextual information in the log message. Usually this would be the id of the object being processed (like the UUID of a particular entity). Human readable information is also welcome.
3. Have a well designed exception system where you ensure which layers do log an error and which layer does the logging of errors. For instance, logging an error in a class called StringUtils is a very bad idea since different callers of this utility class may want to handle errors in different ways. Also, it is hard to determine which process had an error with the StringUtils. In addition, you usually do not (and you should not) pass contextual information to a StringUtils method - which is recommended to include in #2.

Further thoughts: I found this great blog post reflecting on how our logs are not human readable yet it is also not computer-readable (since the software stack like ELK is popular).

Beyond TDD

 Today I was writing a class JUnit test code that of course runs very slow as it starts up by bringing the whole AWS infrastructure online (exaggeration) just to run several assertTrue statements..

As you might suspect, I am not fond of writing this test code. This is because I know in my heart that this test code will be more burden than useful:

1. The test will contribute to a test suite that will take 5 hours to complete
2. The test is very hard to read because it deals with CSV importing code.. and Java does not deal well with such string intensive inputs
3. The test has a lot of test specific code which adds maintenance burden
4. Dev only test code, modern test code strive towards making it readable for non-technical

This frustration brought me an idea. A vision of testing Beyond TTD. What if tests live inside the application and can be used run time? The main motivation for this is to really have a living test suite and documentation. This makes it so that non-technical users can add and maintain test cases themselves with normal user-interface.

It has pros and cons of course, but it feels to me that this is the right balance. You will of course needs to really design this and probably favor pure functions to aid in test simplicity..

PS: The title is inspired by Nico Rosberg's podcast called "Beyond Victory"

TIL: The JVM can ommit stacktraces in the name of optimization

While investigating an issue, I saw a NullPointerException without a stacktrace..

ERROR logs that omits the throwable object (and it is available) is quite devastating. I call such situation 'swallowing the exception'.

However, this time, it is very different. Apparently the HotSpot JVM may opt to not log the stacktrace but  only the exception:

source: https://stackoverflow.com/questions/2411487/nullpointerexception-in-java-with-no-stacktrace

Having a look at older instances of this ERROR log, I found that the occurrence the day before indeed had stacktrace. This is quite surprising and something I would not expect..

Software Methodologies and Religion

I have recently read an interesting 2006 post from Steve Yegge titled Good Agile, Bad Agile. The comments section has also informative criticisms.

It's good to read and understand various ideas. From my experience and knowledge so far, I think the biggest problem with methodologies is that company X can be so different and it is very easy to blindly 'adopt' a methodology without putting in the effort in understanding it. The same with religion and superstition. (Idea I got from this article).

The most interesting idea I have learned so far on this topic however comes from Edgar Schein. Paraphrasing the idea: "Professional Relationship hurts Communication". I highly recommend listening to him on YouTube. I look forward to read his books.

TIL (Today I Learned) - various important things for Software Development

1. When resetting a whole column on a relational database table, dropping/adding the column might be better than running an UPDATE. In my case, I had to update two columns to NULL from a table with 35 million rows.
2. OAuth 1. I had to support a third party to use our REST service that uses OAuth 1.0a. In only a couple of days, I had to learn how this spec works and write up the details needed by the 3rd party to write a client. I'm glad that today there's plenty of resources for this complicated spec; I can imagine that this would be very troublesome to understand in the past. At least a former colleague had a lot of issues when he worked with Facebook OAuth in the past. This blog post really helped me out in quickly understanding OAuth.
3. Monitor HTTP traffic of java programs via Fiddler. With networking setups, it is often easier said than done. By experience, I knew I would be in trouble with how to setup the SSL certs, force java to to use a proxy server, etc. Surprisingly, this is actually easy to do. I use this guide and was able to set it up in 5 mins!

Print Plain SQL Select Statement in Slick 3.0.0

If you were searching the net on how to print the extremely useful plain SQL select statement when using Slick 3.0.0 queries; Then it's your lucky day.

Here's a working example wherein AccountActivityHistory_00Table is my table.

  val q0 = Replica.AccountActivityHistory_00Table.filter(row => (row.DateCreated >= start && row.DateCreated <= end) || (row.DateModified >= start && row.DateModified <= end))
  println(q0.result.statements)

It's in the result field, as in result.statements, this is an Iterable of Strings.

As of this writing the example from Typesafe isn't updated yet to slick 3.0.0 release:
https://github.com/typesafehub/activator-hello-slick/blob/slick-3.0/src/main/scala/HelloSlick.scala

And the upgrade guide doesn't tell us anything either:
http://slick.typesafe.com/doc/3.0.0/upgrade.html

Workstation Automation I: Startup Tomboy Notes

I use Tomboy Notes as my paperless way to track my working hours for almost a year now. Everyday, when Ubuntu finishes booting up, the first thing I do is to start two applications: Firefox and Tomboy Notes.

Screenshot of my May 12, 2014 worklog

Once I Tomboy notes starts; my next step is to "Create New Note", then type in the date today as the title of my work log for the rest of the day. And last, put in my time-in.

This only took my around only 10 to 15 seconds of my time everyday. Small amount of time; however, when you add the numbers for a year, it adds up to an 1 hour (262 Working days - ~20 holidays * 15 sec) + the precious cognitive load when you do this + thinking about "why can't I automate this".

Thus I finally decided to automate this and I was able to reduce the time spent to 5 sec + work satisfaction.

Here are the steps I did: By the way, I'm using Ubuntu 11.10
1.  Use "Startup Application Preferences", add in the applications you want to start automatically. In my case, Firefox and Tomboy.

2. Adding firefox is easy. However, for tomboy I need to use xargs to pass in the date today as a parameter. And for this work, I used an sh script.

#!/bin/sh

date "+%B %d, %Y" | xargs -0 tomboy --new-note

3. Learning to use the date command should be easy. xargs however is little more difficult. Fortunately, this page explained it well.

4. Save this script as tomboy.sh, then make sure that it can be executable at boot time:

chmod 777 tomboy.sh

5. Enjoy!

Me doing a talk about Eclipse at Philippine Tech Hackers meetup

Me doing a presentation for phacker's hackmakati, topic "Editors & IDEs". My presentation was for Eclipse.



It's been too long since I did a presentation like this, too long that trees can already talk. Well, specifically 2 years already.

The audience was a mixed, rubyists, groovyists, perlites (or stringmaster, as suggested by a good friend/buddy from ##programming), a few droids & even non-programmers. I hope these folks didn't get bored with me speaking in English. Yep, there were non-tagalog folks so I had to talk in English - something I did not anticipate, because the last time I went to phackers meetup (2 months ago), we were very few & all filipino.

The other talks (vim, eclipse, sublime) were great! their presentation stacks were a lot better than mine :|. At the very least, I got to show this cool image:



Lesson learned: my english & confidence skills (if they were existent) went really bad. On the positive side, it was a great experience, exp that you do not get by everyday. Thank you Philippine Tech Hackers!

Link to my presentation slides.

/* also thanks to my good friend Fuji for lending me his laptop */

Lisp, A Brave New World

Well, at least for me, Lisp is a "Brave new world"

I finally got the urge to learn Lisp after reading Paul Grahams "The Blub Paradox".
I'm surprised to say, "Nothing beats the feeling of accomplishing Hello World".
The last time I felt this was with C (using turbo c). Not sure with Java's hello world though XD.

Achieving this is just 2 steps:
1. Download 'Lisp in a box' from http://common-lisp.net/project/lispbox/ and unzip.
2. Run the launcher and type "Hello World".

A long shot, JTyrian

 It has been years since I have done game programming. Back then it was J2ME or with Swing using a lot of JPanel. I did attempt to learn Java 3D or those popular game programming libraries like JMonkeyEngine, but nah, all I came up was HelloWorld.

But the wheel of time has turned & now there's LibGDX. It's great, IMO, documentation for beginners is not so great. Luckily I found this blog that has a series of tutorials.

So far so good, I've made the program work up to post #3. The source code is @ revision 6 & is open source here.

Another primitive File I/O gotcha!

A part of the feature I'm writing is to load multiple regex from a file. The file is simple, each line represents a regular expression.

\b(cat|kitten)\b
\b(dog|curr)\b
\b(buffalo|tamaraw)\b

Now I needed to test my method that uses these regex:

public class Animals {
private List regexes;
public Animals(List regexes) {
this.regexes = regexes;
}
public String detectAnimal(String input) {
// for each regex, create a java.util.Pattern and at the first match, return Matcher.group()
}
}

The test data for animals is around 1 million that is stored in DB. For me to be able to test this is to use Groovy's DataSet to loop through the test data. For each iteration, run the detectAnimal method and save to DB. Easy enough.

Now here's the big Gotcha. Below is the Groovy code I used to build my List of regexes. Note that I used this code only for testing purposes.

File file = new File(Animals.class.getResource(Animals.DEFAULT_CONFIG_FILE).getFile()) Animals animal = new Animal(Arrays.asList(file.getText().split('\n')))

A character '\r' was not removed from the resulting list of regex which looks like this:

\b(cat|kitten)\b\r
\b(dog|curr)\b\r
\b(buffalo|tamaraw)\b

This essentially breaks all the regexes except the last one: \b(buffalo|tamaraw)\b Debugging this (I used Eclipse) was tricky since '\r' or carriage return is non-printable character.

To make this much more complicated, in the Java code, I have already written the code that loads the config file which uses Apache Commons IO FileUtils. This code was located in a Project wite Utils class which uses a java static block to load config files. Which, honestly, I don't like. Since using static blocks is a bad practice.

Honestly, this might have clouded my judgement of not using this Utils class. Also since Groovy is a shiny new language, so I was excited in using the new & improved File class. But apparently, Groovy's File class does not have readLines as opposed to FileUtils from Apache Commons.

TODO, make this post more readable

Pokémon Exception Handling

from http://www.codinghorror.com/blog/2012/07/new-programming-jargon.html

Pokémon Exception Handling

woot4moo

For when you just Gotta Catch 'Em All.

try {
}
catch (Exception ex) {
   // Gotcha!
} 

Rivals Schools' NEW ALBUM is here! got worn-out in coding, I'll enjoy music for now :)

Rival Schools, a New York based Post-hardcore band, one of my favorite underrated music. For me, Walter Schreifels (vocals & guitar) is a gifted music genius who writes good lines & riffs.

If you are a fan of underrated music, or looking for one, this is definite constant, no exception.

They're new album titled "Pedals" is a long-awaited album for fans, after the former "United By Fate", Walter has been busy with his solo album and side projects.

Track listing

1. "Wring It Out" – 3:28
2. "69 Guns" – 3:22
3. "Eyes Wide Open" – 3:00
4. "Choose Your Adventure" – 3:25
5. "Racing to Red Lights" – 4:03
6. "Shot After Shot" – 3:14
7. "A Parts for B Actors" – 3:38
8. "Big Waves" – 2:59
9. "Small Doses" – 3:59
10. "The Ghost Is Out There" – 3:20

Personally I like '69 Guns', (great live performance below).
live@ spinner's The Interface, austin, tx


see the lyrics here

You can also listen to purevolume for the Album's single & an exclusive live.

sources: wikipedia.org, wikia.com, spinner.com

A Junior Java Software Developer Diary # 2

Funny day long of coding. While we (me, justin & gladys) our team does our very own daily conference (chat) methodology :), I got caught typing a jejemon style word. "paupdatezzzz", was just me telling that I'm already tired.

I am neutral with all the kill em' all jejemon buzz, but it got interesting and funny. 

There are already a lot of jejemon stuff including a wikipedia article and a cool http://www.jejeschool.com :) And as for my idea, jejemon programming language!

I already made a sample for the language construct :D Happy coding!

A Junior Java Software Developer Diary

A Junior Java Software Developer Diary

Since around a month ago I've been learning a ton of information from a Godlike Java Architech ;)
So before I forget some of these priceless lessons & experience, I'll  be making a series of log about it :0.

Everything from real-world requirements gathering, proposing a design, architecture design, tools to everyday coding.

I'm really feeling lucky ;)

My Android App in Progress!

Finally got my first Android App in progress!

thanks to Chris (http://www.kreci.net/) for the inspiration. I'll describe more of this on my next post, I need to sleep now, it's 2:23AM already.

Black in Black

I'm back from the depths of the void ;)

Very long since my last post here. So much have happened, unfortunately undocumented.

Fortunately, I've found a new sense of importance in logging (web). Three things.

(1)Most importantly is to have a record of your thoughts. Because definitely tomorrow will be another big day and it's likely that you'll forget most of what happened today.

(2)Next is to share, well I hope my posts do help someone out there :D

(3)And last is to monetize, though commonly thought as the root of all evil (it can), it's also an important thing for us humans to live.

Right now, so much is happening, so much stuff I'd like to todo:

1. Android
2. Make a great Blog
3. My fulltime Software Engineer Job (busy)
4. Prepare & Sell my PC

I'd like to attribute this comback post to AIC's comeback album BGWTB. I'm happy I am able to log again here ;) till next post!

JavaFX and Lars Ulrich of Metallica!

After finally appreciating JavaFX, and now maybe I'll say that JavaFX is damn cool because of this app I just found.

http://www.javafx.me/crudfx/examples/RhytmBox.html

It's a simple JavaFX application, but for Metallica a fan and a Java programmer, this is awesome, haha!

Way better than Winamp (maybe but it's FOSS)

A new player that I use. Better than teenspirit/jaangle and (maybe) way better than Winamp :D
The player is called [b]aTunes[/b].
Website: [link]http://www.atunes.org/[/link]

Quick Overview:
1. Open Source (GNU General Public License).
2. Cross-Platform (Written in Java, my lang of choice :D).
3. Last.fm support (MAL for music fans).
4. Built-in lyrics viewer and album-cover downloader.

Written in Java, it also uses Substance, an open-source super cool Look and Feel Java which I used before :D