Saturday, February 11, 2017

About last year...

It has been a while since I last posted on this blog, so I guess it is time for me to update it again and register some of the things I have done last year (2016).

This past year was a little different for me than the previous ones, as I have tried to do things a little differently. Instead of working full-time in a company, I have decided to focus on my own projects, while working part-time as a freelance developer and while studying as a graduate special student at University of Sao Paulo.

Personal projects


Thanks to this change I had the time to work on apps like inSPorte, which allows people from Sao Paulo to see where the buses are in real-time, and it also allows them to rate and report problems with these buses.

Unfortunately this app hasn't grown as much I expected it to, but this experience has taught me that developing an app is only part of the story and that a lot more has to be done for it to be really successful. It has to be well advertised, it has to have a well defined business model, etc...

Off-topic: Since I was mentioning a public transit app, let me announce that I have just released a simple app called Nexbus that allows people from Dublin to check when the next bus is arriving in a bus stop.



Freelance projects


As a freelance developer I have been working on projects such as Bike da Firma (now Bora Bike) and SportsToGo. During last year I have also worked on a virtual reality app called Honda VR Experience.

Honda VR Experience

The car maker Honda was about to release a new version of their Honda Civic car in Brazil, so they ordered a VR app to promote this new vehicle. This app was meant to present a virtual test drive of the car (in the form of a 360º video) before it was released to the public. For this campaign, Honda bought around 250 Samsung S7 + Gear VR, installed this app on each of these devices, and then sent these devices to every Honda car dealership in Brazil. Needless to say, Honda had a very big budget.

Some frames of the 360º video and scenes from the app
The app as viewed on the Gear VR

Configuring and installing the app on multiple devices was also part of the job

That was my first time working on a commercial VR app, and it was also my first time working with an advertisement agency (Jüssi) and an animation studio (Big Studios). My role in this project was to develop the Unity app for the Gear VR: writing scripts for the scenes, UI elements, controls, and for the playback of the 360º video. However, Jüssi and Big Studios were the ones who produced all the visual assets, including the 360º video. This app also used a special plugin from Two Big Ears for the playback of the 3D immersive audio produced by Upmix.



I definitely had some fun working with VR



Graduate studies


I studied "Artificial Intelligence" and "Laboratory of Image Processing and Computer Vision" from August to December. While studying "Artificial Intelligence" I implemented some classic AI algorithms in Python, such as heuristic search (A*), Minimax, decision tree classifier (Hunt's algorithm), MDP (value iteration and Q-Learning). However, the Computer Vision course required us to do some real research and to come up with new ideas to try to improve some state-of-the-art algorithms for problems of this field. Clearly, one semester is not enough time to achieve real improvement, however we have got some interesting results in the problems we have tackled.

This course had only two projects, the first one required us to use the Image-Forest Transform algorithm (which is a generalization of Dijkstra's algorithm for finding shortest-paths in a graph) together with other image processing techniques to create a program capable of correctly identifying connections, drawn by a person on a sheet of paper, between a set of dots. The idea was to use this program as a tool for a smarter automatic grader, allowing students to answer multiple choice questions by connecting dots instead of filling squares.





One of the great challenges of this project was to create a path-cost function for the Image-Forest Transform that could correctly detect overlapping connections. In my implementation, darker paths would have a smaller cost than lighter paths and smoother paths would have a smaller cost than paths with an elevated curvature (i.e. with sharp turns), so these smoother and darker paths would be favoured in regions with overlap. This option works well in most cases, however it can fail when the person only draws connections with sharp turns instead of smooth ones. A solution for this problem would be to use this path-cost function that also analyses the curvature only in regions with overlap, instead of using it for the whole image, so that regions with no ambiguity that have sharp turns would only have its brightness analysed and not be affected, but to do that it is necessary to create another algorithm capable of finding regions where connections overlap.

One does not need to know Portuguese to understand that there's a lot to be improved
The second project required us to use a "3D version" of the Image-Forest Transform algorithm or a Graph Cut (Max-flow Min-cut algorithm) to implement a program that performs a "skull stripping" (i.e. automatic segmentation of the brain from 3D MRI images). In my case I have used both algorithms, so that I could compare both runtime and quality of the segmentation generated with each algorithm.

In simple terms, the Max-flow Min-cut algorithm runtime depends on the weights of the graph, so for 3D images with the same number of voxels it can still vary a lot, however it is capable of generating a segmentation with smoother edges when compared to the Image-Forest Transform. 

The first step of the program finds a region of the image with high average brightness and low variance

Get in touch with me if you are interested in reading my detailed report (in Portuguese) on any these projects.

That's all for now! Hopefully I will post again before the end of this year.