Udacity Self-Driving Car Engineer – nano-degree review

TLDR: my overall rating the course is about 7/10. It was fun doing it, I’m happy I completed all but I’m also glad I didn’t pay for it out of my own pocket.

  • the good:
    • (1) gives wide understanding of many components that make AV architecture
    • (2) a lot of material that requires quite some effort to finish (tho clearly not exhaustive or comprehensive of AV domain / state of the art)
    • (3) for most part well maintained and professionally produced learning material
  • the bad:
    • (1) at times very uneven quality of course material
    • (2) half-year completion time for somebody doing this after hours is very very understated
    • (3) doesn’t push too deep into any particular topic (this could be good ;p)
    • (4) practical skillset only as good as it can get from academic course (i.e. doomed to be outdated; in practice never enough for very competitive industry)
    • (5) the marketing spin on how great this course is for job-finding is a bit annoying (and imho obviously untrue)
    • (6) course structure could be better

Udacity nano-degree?

About half a year ago I started a course from Udacity on Autonomous Vehicles. It’s labelled a “nano degree” and (in theory) takes about half-year to finish. Essentially, it’s a glorified online course with videos/text/coding exercises that are supposed to introduce the student to AV area. Kind of like a full semester course at university. Just that less theory, more practice, yet still packed with material and has long time-frame so ‘nano-degree’ is sort of a justified marketing stunt 😉

Udacity is among the big online learning platforms such as Coursera or Skillshare but its founder is known for his contributions in AV research area and has background in robotics so I always looked at this course with interest. It’s been online quite long actually (since 2012) but I didn’t want to pay for it myself before. When the opportunity came along and I got it sponsored I decided to go ahead!

Why do the nano-degree?

For me, the goal was to do something interesting after hours. The entire self-driving hype works a lot on my imagination and I’m a believer that sooner or later it will materialize. I wanted to see how in practice the technologies I deal with every day professionally (machine learning, deep learning) are applied for robotics and AV. Also, to check out hands-on if any topics from this area will feel right for me and if any would be interesting to get into long-term.

Course overview

Course consists of two big blocks: (1) first is more vision/ sensor oriented; (2) second towards robotics and steering the car. Within each block the general structure is some lessons consisting of text/video tutorials mixed with small coding exercises. After every few lessons there is a bigger project that involves more coding to solve some practical AV problem. Frequently, the coding does not really start from scratch – there are some source code templates and the students needs to fill out some missing functions. Overall, for an online course I would say that’s as good as it gets.

Screenshot from Highway Driving project with trajectory planning.

My general gripe here is that the entire course should be preceded with a module that gives an overview of entire domain (ie. car subsystems) and connect projects to individual blocks. This is informally done at the end but very briefly. Also, I think each learning module should have some short initial lesson giving brief state of the art overview – unfortunately that is missing (aside of research paper links but without any useful/ curated explanations). Ideally, as a student I would like to know what’s the current cutting edge; what’s used in the industry/academia and how my implementation relates to the state of the art. It’s obvious that 6-month course cannot cover the most advanced techniques but some executive summary of solutions landscape would make things a lot more clear.

Project assignments review

Below I give my own subjective review of all projects in the course. This can give a high level idea of what the learning materials pertain to. I also provide GitHub links to my solutions for every project.

NoProjectRatingTechnologyComments
1Lane Detection7/10– Canny edge detection
– Hough transform
Objective: Given images/video of road detect lanes using Computer Vision approach

-A lot of fun given this was the first project. Lots of tweaking and adjusting.
2Adv. Lane Detection8/10– Canny edge detection
– Hough transform
Objective: Harder version of previous project. The video contains road with shadows, light reflections, big curves, obstructions etc.

– At some point starts getting boring and tedious all tuning but still feels like learning interesting things and there is a sense of accomplishment as tasks are more difficult than previous project and results look better too.
– (*not a course drawback but Computer Vision) Overall one thing that feels futile is how fixed all the settings are accounting for specific conditions of the videos (and the thought that with new video it would break).
3Traffic Sign Detection7/10– Machine Learning
– Deep Learning
Objective: Typical image classification task. Given road sign photos, detect which sign is it. This skips the object detection part (ie. bounding boxes), images of signs are already extracted from the photo. Goal is to achieve certain accuracy.

-Decent amount of own coding and tuning. Quite fun project. I would probably rate higher, if i wasn’t biased with doing ML research as my day job.
4Behavioural Cloning 6.5/10– Machine Learning
– Deep Learning
Objective: Given road curvature angle decide what should be the steering angle of the car. This is done with deep learning, goal is to drive car around a track without going out.

-A lot of copying, little tuning for the basic track. Similar as previous project, I’m probably a bit more biased here in comparison to regular Udacity student.
5Sensor Fusion3/10– Numerical Methods
– Signal Processing
– Kalaman Filter
Objective: Estimate position of moving objects around the car based on measurements from radar and lidar.

-Copy-paste all the way, the lessons before were too much math and not very well explained. The project has a lot of already completed code, the simulator and server clunky to run/test/debug.
6Kidnapped Vehicle6/10– Numerical Methods
– Signal Processing
– Particle Filter
Objective: Locate the position of moving car, given distances and positions of objects from car surroundings.

– Project is fun to implement but the explanations of how to implement Particle Filter are poor, I think could be done a lot better. The code template and variable naming inconsistent and makes coding harder rather than easier. The project code seems to have been updated over the years and is inconsistent with the course instructions.
– (*not quite course issue) Similar as Kalaman Filter, the Particile Fitler is a numerical method, therefore bug tracking can be a nightmare, small issues cause the entire solution to go way off.
– Finally the simulator is a resource hog, in previous projects it didn’t matter much but in this project, there is a time requirement. Therefore, running the simulator and algorithm on the same machine causes a lot of bias.
7Highway driving7/10– Path planning
– A star
– Dynamic Programming
Objective: Calculate car speed and trajectory during highway driving scenario, including lane changing.

– Lessons leading up to this project seem very inconsistent (e.g “Lesson7: Search” – assignments and solutions are out of sync)
– Aside of the drawbacks I found this project fun, it the simulator worked better I think that project would have a lot more potential.
8PID controller5/10– PID controller
– Twiddle algorithm
Objective: Implement car steering (ie actuation), this is same task as project 4 but non-ML approach.

– The start of the project was interesting, especially to see how very very few lines of code manage to drive the car in track!
– Afterwards I spent quite some time on improving the initalizaation and implementing Twiddle but that did not give too much effect, which was quite demotivating
– Overall, this project feels a bit like the Lane Detection: so much depends on some “magic” parameter settings that it feels a bit impractical.
9System Integration2/10– Implement ROS packagesObjective: Drive car on a test track. Implement some small bits of various car subsystems.

– In nutshell I find this project is less on algorithms more on learning how to code using ROS framework. There are some more technical bits too that add to difficulty, especially if one didn’t follow Udacity walkthrough and wanted to do some optional material like traffic light classifier.
– My overall impression is this was sadly the worst project assignment of them all. Mainly due to big hardware requirements and therefore big problems running ROS and simulator side-by-side. In previous projects I already had problems with simulator now my hardware being additionally hammered by the ROS – everything is extremely frustrating.
– The only fun/interesting part was learning a bit about this ROS framework in a hands-on tutorial fashion which went beyond the regular ROS Gazebo simulator.

Overall, I think the course quality is best for initial modules. I had most fun with with both Lane Finding projects (tho all the parameter setting frustration tells me old school CV might not be the thing for me). In line with my professional interests both machine learning projects were quite cool too, although I felt like “not enough”. Finally, from second part, I liked Highway Driving project a lot, sometimes frustrating but a lot of fun. After completing felt like I could dig a lot more into this.

If i had to point out the single biggest gripe of all project – the simulator. It’s quite bad. I’m guessing the intention was to make projects more visually appealing but due to performance issues this ends up being very very frustrating. I was running the simulator on either Macbook Pro or Surface Book 2 with GPU and super annoying on either (and its not like the simulator has some ‘next-gen’ graphics).

Wrap up

I suppose the best recommendation I can give this course: in comparison to regular academic courses that I had during my university years , this seems just about there in terms of quality. Lecturers are competent and learning material is quite decent; albeit lesson to lesson that quality goes up and down, especially the project assignments. I think this is due to some modules being refreshed after years and some being old ones.

The marketing spin about amazing skillset the course gives is blown quite out of proportions. I doubt anybody believes the advertising Udacity does related to landing a great job after this course but it’s a bit annoying to keep on listing to it throughout the course. Likewise, the lecturers giving some odd over-the-top appraisal on how big accomplishment it is to finish an implementation of this or that algorithm. It’s obvious why Udacity does it but I find it unnecessary and irritating.

One rather demotivating bit that I started to notice about middle of the course – the learning material (videos, text, coding exercises) end up not contributing to the project that comes after. Sometimes there is quite an in-depth study of some algorithm with lots of maths etc but the project that comes after ends up completely not using it.

Another issue: finishing the course within the recommended half-year time requires a lot of effort. I think it’s quite impossible if you do the course after work, maybe ~2-3 hours a day. Quite a few times, I decided to skip optional assignments because I knew I wouldn’t be able to finish the entire course on time otherwise. Likewise, I totally wanted to go deeper on some subjects that interested my but had no time. You could say I could just take it easy and go past the 6 months mark but the trick is access to the course is time-based – after the initial 6 months, you need to pay again to have full access 🙂

IMDB text classification: keras vs tensorflow 1.x vs tensorflow 2.0 vs pytorch

When trying to learn coding some deep learning problems, you can bump across quite a few (python) frameworks. Personally, I started with Keras, being the easiest, but then I was curious how others work.

So, as an exercise, I implemented the same problem in several frameworks trying to make it work exactly the same (as in take same input, give same output and implement the same kind of neural network inside).

The starting point for me with Deep Learning in general was quite a good introductory book “Deep Learning for Python” by Keras author Francois Chollet. Therefore, my Keras code follows instructions from the book, while the subsequent implementations in other frameworks I did myself trying to mimic what’s done already in Keras – small exercise in getting to know different APIs 🙂

I used the IMDB review classification problem (trying to figure out positive/negative reviews given their text). Implementing a simple MLP network with 2 hidden layers.

I tested the following, implementing the same IMDB problem in all:

All source code is available in Jupyter notebooks on GitHub, I ran it using python 3.7.

My general personal thoughts are that Keras is by far best for quick prototyping and doing many many experiments after which you might need to revisit some again (ie. code is simple and readable). At work, I often need to test many cases/architectures/configurations for a client, jump to another project and maybe few weeks later come back to my old experiments to continue or pick something for final deliverable.. so readability of code and being able to quickly recall what I actually did is nice.

Of course, I looked at other frameworks for a reason. Keras is high-level, so when I need more flexibility and control to implement something custom then might need to go elsewhere. The almost legacy now TensorFlow 1.x is truly a pain to debug, very unintuitive in comparison to any other python framework/library. PyTorch has a more interesting approach but I found it not as good documented and after learning both Keras and TensorFlow 1.x quite annoying actually :p Finally, TensorFlow 2.x is basically trying to steal the best tricks from everybody, fully integrated Keras and than for more control it adopted PyTorch-like approach with defining the model through subclassing etc. At the moment of writing this TF2 is still beta / release candidate, so I would say needs time to mature (e.g. I’m getting lots of warnings when running under Windows even the basic TF2 subclassing example, the documentation even tho improved I still find lacking etc.). Overall tho, I think its a good direction.

ps. my opinion could be a bit biased towards Keras because by the time I started with TensorFlow/PyTorch I already went through the entire Chollet book implementing everything and I used Keras for some problems at work with clients too.

After hours: NAFA Visual Communication course review

In my earlier blog post I gave a short description of a graphic design course I signed up for at NAFA, Singapore. Check that post if you wanna know about the content of the course, organization, scheduling etc. In this writeup, since the course is already over, I’ll just add on some new thoughts on course quality!

The academy gives rather generic information about the content of courses and I couldn’t find a review anywhere online, I figured I’ll write my own impressions about the course and how useful I found it (in case somebody wants to sign up in the future and wondering if its worth it).

What I was looking for (quite a lot ;p):

  • Strong theoretical principles of design and related aspects. I already had some practice so I mostly wanted the academic knowledge (usually the stuff everybody wants to skip when doing a degree but I guess perspective changes when older ;p)
  • Learn the science on how do humans perceive colours, what colours (and why) go well together and others don’t, similar about shapes, distance, perspective. How light changes the perception of all those things etc.
  • How do shadows, shading and reflections add depth to design etc.
  • What are different schools/ historical trends in design. How to compose a design (how should a scene look like and what effects do we get by including certain compositions), how does design effect vary depending of objects in the scene (humans, animals, items ?)
  • What are the typical errors and achievements coming out of implementation out of all of those…

What I got:

  • (very) little theory
  • lot of tips and practical knowledge from experienced pros
  • beginner Illustrator/Photoshop tutorial

So comparing my expectations with reality – mostly the course failed to deliver on what I originally wanted and I got little answers to the questions I had. Having said that, it’s not all bad: the course was at times somewhat interesting in the direction it went and I learnt few of things I didn’t know.

The biggest benefit in my view: the teachers push you to work on projects at home as there are a lot of assignments (good!). Furthermore, personally for me, somehow it was a starting point to investigate literature and study the things I wanted to know on my own ( so in other words: bummer for the the course but at least I got motived/pushed into the right direction).

Outside of the fact that the course missed my expectations, I have a feeling it’s rather bad value for money (even tho it’s not particularly expensive). During Visual Communication 1 (Principles of Design), most classes started late and actually teacher had very little to say, so finished early as well. Furthermore, after the few initial introductory classes the majority of this course is just consultations. Teacher approaching each student individually and giving his feedback on the project progress that student develops at home (basically you get ~5 min chat out of what’s supposed to be 3 hour class).

Visual Communication 2 (Digital Design) is more rich in practical knowledge, at least at the beginning. The first 5 classes are packed with learning Illustrator tools one after another, afterwards unfortunately it turns into the same as Principles of Design. Come to the class and teacher basically tells you to do your work (and ask questions if you like).

During the second semester, Visual Communication 3 (Typography & Colour Study), we got surprisingly a bit more theory and slides than VC1/Principles of Design (like I wanted ;p) but unfortunately I think it lacked a form of short in-class exercises that would somehow help to sink in what teacher talked about.

Finally, Visual Communication 4 (Digital Design 2), is exactly the same formula and impressions as Digital Design 1. Except that instead of Illustrator tutorial, it’s 2 classes on Indesign and 3 on Photoshop.

To sum it up you can check out the results of my homework assignments in my portfolio page (and also compare to few projects I could publish and did before the course):

Final note: during a single year NAFA can have multiple classes for the exact same course and those are run by different teachers (usually they do it part time too ;p). From my conversations with students from other classes, it turns out that this can make quite a difference (different home assignments, slightly different exercises/curriculum in class).

After hours: NAFA Visual Communication course

I’ve been doing all sort of design and drawing for quite a while now. For fun, privately but also for my projects at work. Web design, DTP etc…

Everything I did, I learned with trial and error and own experimentation, without doing any proper courses or studying design at a university. Recently, however, I thought it would be a good idea to get some theoretical background to boost whatever practical design skills I got throughout working on my projects.

So … I signed up for a part-time course on Visual Communication at the Nanyang Academy of Fine Arts, Singapore. It’s a rather old and renowned public education institution in Singapore. The course has an interesting curriculum and allows to have a day job + attend classes in the evenings. There are 4 parts to it:

  • Visual Communication 1 – Principle of Design (10 sessions x 3 hours)
  • Visual Communication 2 – Digital Design I (10 sessions x 3 hours)
  • Visual Communication 3 – Typography & Colour Study (10 sessions x 3 hours)
  • Visual Communication 4 – Digital Design II (10 sessions x 3 hours)

Parts 1 and 2 go concurrently for about 2 months and afterwards parts 3 and 4 for another 2 months.

The academy gives rather little information on what the courses contain, so in the reminder of this post I’ll explain it from the participant point of view and in another post I’ll update with my review on the entire course once I’m done with it.

“Visual Communication 1” main theme is logo/business card design and teacher passing some of his professional experience with addition of little theory (backed by some minimalist slides).

“Visual Communication 2” is basically an Illustrator tutorial going through some most typical functions (and exploring them from different angles) and concluding with work on a packaging design project.

Each week those two run concurrently but are not directly related (ie what you hear in class on principles of design is not practiced later in Illustrator class), however in totality of things logo design is done in Illustrator, so the lab class somehow supplements the logo assignment.

After hours: Portfolio publishing – CMSes review

I made a little review of dedicated content management systems for publishing online portfolios (e.g. for designers etc.). Be advised – all this info is fully subjective.

I did this for a (web designer) friend and my judgements are done based on “how easy it is to setup the portfolio” vs. “how much you can customise it” to look individual and make the portfolio itself a showcase of ones design skills 🙂 Also I looked on “how easy it would be to add new projects”.

Publishing this in case somebody like me is looking for a list of such systems.

Name (+my rank ) Real (Cool) Examples Short Review
— CMS with Hosting —

Cargobad

http://butdoesitfloat.com/

http://www.funkhausdesign.com

http://cargocollective.com/seiz

Fairly rich but only their hosting. The free version is quite
limited (number of projects, images etc.). In my opinion unless you really like this, own hosting is better.

4ormat.combad

http://martynagalla.com/

http://hollysuangray.4ormat.com/754-holly-suan-gray

http://mason.4ormat.com/278-photography-home

 

Similar like Cargo but more simple.
— Need own Hosting —

Indexhibit

http://www.useyourfriends.com/

http://www.ty.sg/hello/

– simple
– different type of gallery layout, different themes
– in the basic out of the box version add projects/portfilio entires
under predefined sections (on-going, projects, Info)

Secretary

http://www.chilifactory.dk/

http://caseybritt.com/work/ajc-unplug

 

More the same direction/type as Indexhibit but probably better
developed and maintained.

Stacey

http://andmelbourne.com/projects/nicolinis/#1

http://helloalan.com/

http://kimberlymeenan.com/

– does not have interface to manage content, everything is just
by copying files- extremely lightweight, it doesn’t seem to have many options
for configuration-
everything is organised by text files with content & HTML
templates that are filled with this

Subfolio

http://400asa.net/

http://mobile.believein.co.uk/

Very much like a online file browser that can be themed rather
than a full CMS.

Indxr

http://christopherhewitt.com/

http://marcus-eriksson.com/

Similar like Subfolio its more a script rather than a full CMS.

Core

http://www.grahamdunn.be/

http://diftype.com/

Looks ok and has good reviews but is not available for public and I
couldn’t get it.

Symphony

http://nilshoerrmann.de/

http://lundark.com/

http://www.haggsgate.co.uk/

Normal CMS, its not designer or portfolio oriented but very
customizable.

After hours: RDF Detective Plugin for Chrome

For quite some time now, I stopped using Firefox and switched almost completely to Chrome. Unfortunately, the letter has much less plugins, including no solid plugin to detect RDFs linked inside html documents.

In Firefox Semantic Radar used to do the job but I got tired of running both Firefox and Chrome when i was working with RDF so….I’ve written my own plugin for Chrome – RDF Detective 🙂

It works almost identical to Semantic Radar, in documentes loaded by the browser it detects:

<link type="...rdf+xml....">

and displays a little clickable icon to open RDF in an external viewer (you can edit the viewer in the plugin options by default I’ve set it to SIOC Browser).

Hopefully, when i get more time I’ll hack the plugin to have more preferences and add few other things. For now, it does its job good enough to use (for me), this is how it looks:

At the moment, I’ve published it only on Google Extensions website, if you feel like testing, head over there and install it. So far I only tested how it works on Mac Os X, running Chrome 5.0.342.9 beta, so feedback welcome!

Update: tested for Windows7 + Chrome 4.1.249.1064. Works as well!
Update2: New version out! RDF Detective 1.3 now supports RDFa detection (renders a different icon and allows to open RDFa in a different viewer then RDF). I also added experimental support for detecting what RDFa namespaces are used and in relation to that render the icon in different ways. Currently 3 options: if there is only Facebook or Opengraph namespace their specific icons get rendered, if there is more annotations of other namespaces then a generic RDFa icon is shown.
Update3: The source code for the extension was published at: http://github.com/cyan-ide/rdf_detective

After hours: Making a FOAF profile for your homepage (part A: the tools)

FOAF is a data schema for describing personal data on the Web and is one of the most known in the Semantic Web world. Basically, its a way to publish information about yourself on the web in form of pure data, just like im doing on this webpage of mine but readable for machines! How to do it and why ? Read on!

Part A: The Tools

So far I’ve only had a FOAF file hosted at my old workplace but now I thought that like every normal(tm) person I should put it up on my homepage. This time tho, apart of the regular personal details I wanted to include all the other data from my page so: publications, projects, link to cv and so on. Simple ? Not as much as you might wish. Here are some tools I’ve tried to make it faster then writing the RDF code by hand:

1st try: Protege. Import the foaf ontology, make the instances and fill out the forms, and thats it! …..but wait not really. Its not too user friendly even for somebody who knows more or less all the Semantic Web technologies…

2nd try: TopQuadrant Composer. Same story, import the ontology, make the instances and…damn its exactly the same story…just a little bit less confusing.

3rd try: Dedicated user friendly apps. So I abandoned the ontology design tools for more simple stuff. Ye some work(tm) and some are even quite nice looking but they kind of don’t let fill out all that I wanted (read projects, publications..). So maybe some other time.

4rd try: “reuse, reuse, reuse”. So finally I’ve decided not to trouble myself with any tools and go back to the stone age and write everything by hand…well almost.

Recipe for writing your own FOAF profile fast(tm): google out a known (Semantic Web) person/geek , download his FOAF file and modify it in an xml editor. It took 30min vs. 3 hours or so trying to find a nice app 🙂

Here’s my profile.

ps. I used Wikier’s (aka Sergio Fernández) profile to make my own. Cheers mate!
ps2. If you are waiting for a earlier promised “why to make your foaf profile”. I have to disapoint you. If you are a regular person that does not have much to do with Semantic Web, then to my knowledge there is absolutly no reasonable practical cause why one should make his FOAF profile…not yet at least.

After hours: Bibliography in WordPress

So while building this page I figured I wanted to have some kind of easy tool to maintain my publications and projects lists. After some searching I found 3 interesting WP plugins:

  • WP-Table Reloaded – adds a nice looking GUI in admin panel to create and manage tables
  • bib2html – generate html directly from bibtex file and put into WP page (unfortunately discontinued and not working in WP2.8)
  • BiblioFly – generate bibliography from its internal base (each resource has to be entered by hand with a GUI in admin panel)

Out of the thee, WP-Table is apparently the best maintained plugin but its not directly aimed for generating the bibliography. Nice GUI, lots of options and CSS customizable.

On the other hand, Bib2html appears to be the best bibliography dedicated solution (especially when used together with i.e. Zotero to organize bibliography). It has good features but hardly any GUI and unfortunately it doesn’t work in the latest WP (2.8).

Than BiblioFly is something in between: like WP-Table has a GUI and internal database but its only aimed for bibliographic references (without bibtex import option like bib2html) and seems in a bit worse shape then WP-Table.

More to come when i decide which one to use.