Since all the mobile handheld devices needed to communicate data amongst themselves, it became apparent that a central unit is in order, accessible by all devices, that could manage & coordinate them, read & learn from independent user traffic samples, and at last share the new traffic knowledge back with the users.
Since our system is meant to be Internet based, we decided to develop an application server, that will run 24/7 on some company cloud services. (Today you can find Busy-Hour serve on http://busy-hour.appspot.com) .
Technology & Tools
Following are the technologies & tools we chose to work with to implement the server:
Google App Engine (GAE)
GAE is a cloud computing platform for developing and hosting web applications in Google-managed data centers.
We chose GAE because of the following reasons:
- Easy to get started - App Engine is a complete development stack that uses familiar technologies (Python/Java) to build and host web applications. With App Engine we could write our application code, test it on our local machine and upload it to Google with a simple click of a button.
- Automatic Scalability - Being built on Google infrastructure, Google cloud services are strong enough to handle tens of thousands of requests per minute by our mobile users on the road. No matter how many users we have or how much data we store, App Engine can scale to meet our needs.
- Inexpensive - Actually, free up to a certain level of used resources. Fees are charged for additional storage, bandwidth, or CPU cycles required by the application.
Python is an interpreter, general-purpose high-level programming language which utilizes OOP (Object Oriented Programming) models.
We chose python for numerous reasons:
dJango Framework
Django is an open source web application framework, written in Python. Its primary goal is to ease the creation of complex, database-driven websites. Django emphasizes reusability and "plug ability" of components, rapid development, and the principle of DRY (Don't Repeat Yourself)
We chose to use a web framework like dJango because adding a web framework over the raw GAE helps in alleviating the overhead associated with common activities performed in Web development. For example, many frameworks provide libraries for database access, templating frameworks and session management, and they often promote code reuse.
Server-Client Communication
In order for the server and the clients so “speak the same language” we needed a common protocol. Therefore we defined a JSON based public API to allow the mobile clients to interact with the server.
JSON
JSON (JavaScript Object Notation) is a lightweight text-based open standard designed for human-readable data interchange. It is derived from the JavaScript scripting language for representing simple data structures and associative arrays, called objects. Despite its relationship to JavaScript, it is language-independent, with parsers available for most languages.
We decided to use JSON for numerous reasons:
The JSON script was then intended to be introduced above the HTTP layer, as the http-packet body content.
Traffic Estimation Process
As was said earlier, during road setup, the web-app installs different roads into the DB. During this stage, each road is divided it into 1 KM length segments, and then stored in the DB. Each segment traffic estimate is monitored separately.
Whenever a new traffic report is received, it is attached to the appropriate road segment it was sent from. If the report is too far from any road it is ignored. Each road segment keeps track of related reports from the past 5 minutes, 10 minutes and 15 minutes. For each time frame, the median speed is calculated from the matching reports set, as the “estimated traffic speed” for that time frame.
We decided to use the Median as the estimator for 2 reasons:
It ignores byzantine samples and overshooting sample (like a motorcycle driving fast in a traffic jam, or a car parked by the roadside of a high-speed road).
It’s easy to calculate from a sorted array (or a DB index, in our case)
After we have the speed estimates for the 3 time frames, we average them into the final traffic estimate for that road segment. A shorter/closer time frame gets more influence in the weighted-average.
- Market popularity - it’s very popular these days, and its handy to know it when applying for a job.
- Cross Platform - so both me and my partner can use it (we work on different operating systems - Mac OS & Windows).
- Google App Engine support.
- Remarkably simple syntax + a large and comprehensive standard library.
dJango Framework
Django is an open source web application framework, written in Python. Its primary goal is to ease the creation of complex, database-driven websites. Django emphasizes reusability and "plug ability" of components, rapid development, and the principle of DRY (Don't Repeat Yourself)
We chose to use a web framework like dJango because adding a web framework over the raw GAE helps in alleviating the overhead associated with common activities performed in Web development. For example, many frameworks provide libraries for database access, templating frameworks and session management, and they often promote code reuse.
Server-Client Communication
In order for the server and the clients so “speak the same language” we needed a common protocol. Therefore we defined a JSON based public API to allow the mobile clients to interact with the server.
JSON
JSON (JavaScript Object Notation) is a lightweight text-based open standard designed for human-readable data interchange. It is derived from the JavaScript scripting language for representing simple data structures and associative arrays, called objects. Despite its relationship to JavaScript, it is language-independent, with parsers available for most languages.
We decided to use JSON for numerous reasons:
- Very popular - Many code libraries that read/write JSON exist for different frameworks and programming languages. Specifically, it is supported on both Android/Java and GAE/Python.
- Human Readable - (as opposed to binary), and hence easy to debug.
- Lightweight - JSON outputs relatively short code (as opposed to other human readable formats, like XML for e.g.). And it is important that the server and client communicate efficiently, byte-wise. Because 3G internet is slow & expensive, server resources are expensive too (reading/writing JSON takes time too).
The JSON script was then intended to be introduced above the HTTP layer, as the http-packet body content.
Traffic Estimation Process
As was said earlier, during road setup, the web-app installs different roads into the DB. During this stage, each road is divided it into 1 KM length segments, and then stored in the DB. Each segment traffic estimate is monitored separately.
Whenever a new traffic report is received, it is attached to the appropriate road segment it was sent from. If the report is too far from any road it is ignored. Each road segment keeps track of related reports from the past 5 minutes, 10 minutes and 15 minutes. For each time frame, the median speed is calculated from the matching reports set, as the “estimated traffic speed” for that time frame.
We decided to use the Median as the estimator for 2 reasons:
It ignores byzantine samples and overshooting sample (like a motorcycle driving fast in a traffic jam, or a car parked by the roadside of a high-speed road).
It’s easy to calculate from a sorted array (or a DB index, in our case)
After we have the speed estimates for the 3 time frames, we average them into the final traffic estimate for that road segment. A shorter/closer time frame gets more influence in the weighted-average.