If you missed Spark Summit, catch this talk from Abdulla about how to build predictive recommendation models and ensure type safety using Apache Spark DataFrames. See how Credit Karma's choice of metric evaluation helps us calibrate models to obtain the best global result, and hear our lessons learned when we scaled our model development environment to handle Terabyte-scale data with thousands of features.
Credit Karma leverages data for over 60 million members to deliver a personalized user experience. To do this, we rely largely on Scala and Akka to do the heavy lifting. Powerful tools, however, demand some mastery on how to use them.
When we were considering how to push 700k events per minute from Kafka into our data warehouse, Vertica, we learned these lessons about how to choose the best framework for high throughput.