Blog

Blog2017-01-30T11:40:33-08:00
1Mar, 2018

Getting started with the Universal Recommender – Single machine setup

March 1st, 2018|0 Comments

The Universal Recommender I’ll assume if you’re reading this that you’re already at least familiar with Universal Recommender, the most popular ML engine for PredictionIO. If you’re not sure what either of those things are we’ll have a separate post on those in the near future. The creators of Universal Recommender (Pat Ferrel and the team at ActionML) make it super easy to get started trying out their algorithm, loading in some of your own data, training, and getting personalized recommendations back out. You can just spin up an EC2 instance preloaded with everything you need and get off to [...]

15Dec, 2016

Using Category Encoders library in Scikit-learn

December 15th, 2016|Tags: , |1 Comment

I recently found a relatively new library on github for handling categorical features named categorical_encoding and decided to give it a spin. As a reminder - categorical features are variables in your data that have a finite (ideally small) set of possible values, for example months of the year or hair color. You can't feed these into predictive models as raw text, so some conversion is necessary to prepare these variables to be useable. Typically, you create a new, separate column for each possible value (or alternately depending on the intended model, n-1 values) and each of these new columns [...]

18Nov, 2016

TF-IDF Basics with Pandas and Scikit-Learn

November 18th, 2016|Tags: , , |7 Comments

In a previous post we took a look at some basic approaches for preparing text data to be used in predictive models. In this post, well use pandas and scikit learn to turn the product "documents" we prepared into a Tf-idf weight matrix that can be used as the basis of a feature set for modeling. What is Tf-idf? Tf-idf is a very common technique for determining roughly what each document in a set of documents is "about". It cleverly accomplishes this by looking at two simple metrics: tf (term frequency) and idf (inverse document frequency). Term frequency is the [...]

24Jun, 2016

A Shiny New Python Data Science Sandbox in 30 Minutes Or Less

June 24th, 2016|Tags: , , |5 Comments

This post will give beginners a full walkthrough to go from nothing to a fully functional linux/python/pandas/scikit-learn environement with jupyter as a front end. For exploratory work, I really like this stack. My native OS is Windows, but since we're using VMs I would imagine the setup for OS X is very similar and probably won't need any modification (other than steps for configuring the VM). If you have a solid internet connection, we should be able to get this all done in under 30 minutes startiiiinnnnnng NOW... 1. Download an Ubuntu Desktop version of your choice. I like 14.04. [...]

20May, 2016

Investigating missing data with missingno

May 20th, 2016|Tags: , |0 Comments

I recently came across a new python package for visualizing missing elements of a data set. The aptly named "missingno" is super useful when you're taking your first look at a new data set and trying to get a feel for what you're working with. Having a sense of the completeness of the data can help inform decisions about how to best handle missing values. In this post, we'll take a quick look at the small and simple Shelter Animal Outcomes data set from one of the current Kaggle competitions. Matrix visualization The first visualization is the "matrix" display. This [...]

10May, 2016

Text Pre-processing Basics with Pandas

May 10th, 2016|Tags: , , , , |4 Comments

In this post, we'll take a look at the data provided in Kaggle's Home Depot Product Search Relevance challenge to demonstrate some techniques that may be helpful in getting started with feature generation for text data. Dealing with text data is considerably different than numerical data, so there are a few basic approaches that are an excellent place to start. As always, before we start creating features we'll need to clean and massage the data! In the Home Depot challenge, we have a few files which provide attributes and descriptions of each of the products on their website. The idea [...]

Go to Top