If told to write a web crawler, the tools at the top of my mind would be Python based: BeautifulSoup or Scrapy. However, the ecosystem for writing web scrapers and crawlers in Go is quite robust. In particular, Colly and Goquery are extremely powerful tools that afford a similar amount of expressiveness and flexibility to their Python-based counterparts.
»Yelp’s MRJob is a fantastic way of interfacing with Hadoop MapReduce in Python. It has built-in support for many options of running Hadoop jobs — AWS’s EMR, GCP’s Dataproc, local execution, and normal Hadoop.
»During the month of December, I used the daily Advent of Code puzzles to teach myself Go.
»2017 was an interesting year. Personally and professionally, I found it to be a successful year of growth.
»This summer, I worked primarily in Java on a backend service for Zillow’s advertising platform. This service was built on top of Spring Boot, an “opinionated” Java web framework.
»This summer, I had the opportunity to do a Software Engineering internship at Zillow. I grew quite a bit as a developer during my time at Zillow, and gained a lot of experiential knowledge in practically applying the technologies that I’ve learned in school and in my side projects.
»I’ve been working on a site for the past month that generates 3D models from Github’s contribution graphs.
After a month of work, it’s complete! So, I’m happy to announce GitTrophy. It’s pretty slick: type in a Github user or repo name, and you’ll get a 3D preview of the contribution model. I 3D printed my chart from 2016, and it turned out pretty well.
»This past week was SGDQ 2017, the latest GamesDoneQuick charity speed-run marathon. For the past year, I’ve been managing an unofficial stats page for these events called GDQStat.us. This is the third event that I’ve hosted this site, and a lot has changed in this iteration.
»AWS is a great place to host static content: Bandwidth / hosting costs are very cheap if you’re at “hobbyist” scale, you get great availability, and AWS gives you free SSL termination / certificate management for HTTPS if you get everything setup correctly.
»I recently started my Summer internship at Zillow. Starting at a new company with a new laptop prompted (forced) me to think about the way I have my development tools setup.
»