Miscellaneous

Data Science: Reality Doesn’t Meet Expectations

Execs frequently ignore data science research when making decisions.

Data is often dirty, or insufficient to make decisions about majority of the users. Sometimes the infrastructure is poor - SQL queries take hours.

Data Scientists are usually the only ‘data person’ on the team. Tons of request from teams, and most of the work is repetitive and ’easy’.

Measuring impact is hard - especially on dollars that were hypothetically saved but never spent.

CityFlo Reduces Google Maps API Bill by 94%

Google Directions API limited their free tier . CityFlo uses this API to calculate ETAs. Initially, CityFlo queried the API every time a stop was registered.

OCR tends to miss (sometimes) really important context.

Make use of derived data, e.g. age at marriage, gender differences, mortality rates, mapping migrations, last names

Don’t spend all your time cleaning data.

Try a sequence of things, not one big idea.

Don’t expect existing tools to do the whole job. All the while, building new tools is not as straightforward.

Sample Digital Humanities Projects

ORBIS by Stanford . A kayak for the Roman Era.