We use big data at Credit Karma for data science modeling and analytical reporting, which helps us assess the efficacy of our products and services. We process and store data in Google BigQuery at scale, and we do this with data security, data quality, data lineage and data management as our key requirements. These make […]
Stopping rules are what make most frequentist (a.k.a. classical) statistical tests valid. Unsurprisingly, they tell you when to stop an experiment, and equally unsurprising, they are rules that must be followed in order to get statistically valid experiments from statistical tests that depend on them. However, despite stopping rules being critical to the interpretation of […]
When I started at Credit Karma in late 2014, we were introducing Kafka into our data architecture. We were building out what would become the core data infrastructure for the company, which is and was growing rapidly. In a mere three months we were pushing 175k JSON events per minute into Kafka. At that time, […]
Originally posted in Scott Livingston’s blog At Credit Karma, my team maintains a service powered by a multi node Akka cluster. Akka is a framework for building distributed, concurrent, and message-driven systems in Scala. It helps us easily scale our service, and gives us some resilience when problems inevitably happen in production. In this post, I’ll cover two […]
