Blog

rustdoc is a great tool, but as of now there isn’t an official way to have its generated docs refresh as you make edits. Running cargo doc with the --open argument will open the generated docs in browser window. If you make changes to your source code, you’ll need to re-run cargo doc to have the changes reflected in your browser. By chaining together a few other Rust tools, we can pretty easily get the functionality of live-reloading docs.

Recently, I was trying out clippy — a linting and static analysis tool for Rust, when I ran into a lint warning that wasn’t immediately clear to me: warning: casting u8 to u16 may become silently lossy if types change.

Over the past few months, I’ve put some thought into tweaking my Twitter usage to better align it with how I’d like to use social media. I’ve never felt particularly “addicted” to or maligned by Twitter. However, left unchecked, it can easily become a time sync that leaves you in a sour mood.

I recently went through the process of switching my blog’s static site generator from Jekyll to Hugo. Both are great tools, but as I’ve updated my website over time, I began to feel constrained by Jekyll — and wanted to switch to something more stable and robust.

This post gives a technical overview and architectural justification for my latest project, corral — a serverless MapReduce framework.

For a recent project, I needed to read data from a specific chunk of a file. The data was a sequence of serialized records, so I used a bufio Scanner for splitting. Scanners are great, but they obscure the exact number of bytes read. In working through the problem, I found a solution that worked quite nicely.

I’ve been going through a period of programming language wanderlust over the past couple months. Recently, I’ve been quite interested in Rust. Coming from Python, I’ve found a lot of Rust’s language features to be quite powerful.

If told to write a web crawler, the tools at the top of my mind would be Python based: BeautifulSoup or Scrapy. However, the ecosystem for writing web scrapers and crawlers in Go is quite robust. In particular, Colly and Goquery are extremely powerful tools that afford a similar amount of expressiveness and flexibility to their Python-based counterparts.

Yelp’s MRJob is a fantastic way of interfacing with Hadoop MapReduce in Python. It has built-in support for many options of running Hadoop jobs — AWS’s EMR, GCP’s Dataproc, local execution, and normal Hadoop.