tungwaiyip.info

home

about me

links

Media

Yucatán Photos

St Lucia Photos

Photo Album

Videos

Blog

< September 2010 >
SuMoTuWeThFrSa
    1 2 3 4
5 6 7 8 91011
12131415161718
19202122232425
2627282930  

past articles »

Click for San Francisco, California Forecast

San Francisco, USA

 

Traffic Data Analysis

I was doing a traffic data analysis base on the vehicle location data pull from the SF Muni website. This is an extremely interesting project. I pick up a whole lot of new skills while doing this, most notably data analysis and computational geometry. The project also turns out to be a challenging one. A few months (of part time work) has gone by I still haven't nearly achieved my original vision.

Still I think I am getting better at this work. So I'm sharing some interesting data I'm working with. On one bus route I've collected 20,000 location readings on a single day. The goal is to organize them into individual vehicle trip. After a first pass, the data are reduced to 336 trips. But how good is the quality of data? And how good is my trip segregation algorithm? So I plotted the trips' distance against their duration on a chart. The graph quickly helped me to evaluate the quality of the result.

Bus #38 trips characteristic

The first impression is the result looks fairly good. A dense band of points shows that most trip length is about 10 km, and they takes about 40-75 minutes. This seems to match real world experience. At the lower left hand corner is a number of trips that last very short time and cover very little distance. I'll probably treat them as noise and discard them. A curiosity is some trips seem to cover a long distance from 15 to 20 km, much long than the official route. Where did they went?

Closer examination on their track reveal the problem. These are all legitimate eastbound trips. However the bus starts from a depot in the middle of the city. They went all the way to the terminal in the west end. Only from there the real eastbound trip begins. The challenge for me is I'm only interested in the eastbound portion. The trip from the depot to the west end is actually data pollution. I need to find a way to handle these extraneous data.

This still leaves a scatter of points of around 5 km long and last 15 to 35 minutes to inspect. Meanwhile I have difficulty to examine the track because Google Earth is crashing on me all the time. I guess this is a good time to turn away from technical work and to write a blog instead.

2010.09.02 [, ] - comments

 

 

blog comments powered by Disqus

past articles »

 

BBC News

 

Brexit: EU demands 'serious UK response' on citizens' rights (29 Apr 2017)

 

North Korea crisis: North in another 'failed' missile launch (29 Apr 2017)

 

Turkish authorities block Wikipedia without giving reason (29 Apr 2017)

 

France election: Marine Le Pen would make Dupont-Aignan PM (29 Apr 2017)

 

Tense moment plane lands without wheel (29 Apr 2017)

 

Italian woman 'returns mixed-race surrogate baby' (29 Apr 2017)

 

Hungary education law: Orban 'to meet EU demands' (29 Apr 2017)

 

El Salvador zoo: Prosecutors investigate 'suspicious deaths' (29 Apr 2017)

 

China deports US 'spy' Sandy Phan-Gillis after conviction (29 Apr 2017)

 

Ted 2017: Elon Musk's vision for underground road system (28 Apr 2017)

more »

 

SF Gate

 

Bay Area News (7 Jan 2012)

 

City Insider (11 Feb 2012)

 

Crime Scene (13 Feb 2012)

 

C.W Newius Column (10 Jan 2012)

 

C.W. Nevius Blog (11 Feb 2012)

 

Education News (10 Jan 2012)

 

KALW (11 Feb 2012)

 

Matier and Ross Blog (11 Feb 2012)

 

California issues historic medical marijuana rules (29 Apr 2017)

 

In dot-com bubble time, it’s still only 1997 for U.S. equities (29 Apr 2017)

 

ICYMI: Trump’s job hard, dirty jeans scorned, Charmin for Butt (29 Apr 2017)

 

Craigslist sues Instamotor for allegedly re-posting car ads (29 Apr 2017)

 

Wait for calorie count on burgers, pizza may get longer (28 Apr 2017)

 

The biggest regrets people have after buying a home (28 Apr 2017)

more »

 


Site feed Updated: 2017-Apr-29 12:00