Building a Better Opencourser, April 2017 Edition

Last week, we received a huge spike in traffic thanks to this post on Reddit's /r/YouShouldKnow subreddit. As a result we were able to:

  • Test a few new features we recently rolled out
  • Quash a few bugs we didn't find in our own testing
  • Make improvements and build features based on feedback from Redditors!

Now with better performance

We noticed a few weeks ago that our site was beginning to bog down, especially after adding our latest batch of 1,500+ courses. Keep in mind, we originally designed and tested OpenCourser with a test set of 1,000 courses. A lot of the code we wrote originally simply didn't scale against the new course count of ~6,400, which made 95% of OpenCourser very sluggish.

GTmetrix results... It took 7.1 seconds to load a page that was primarily text!

That meant we were asking our servers to do a lot of work to put together each page that our visitors would see. If you visited our main landing page, for example, our Python app would spend about 3.5s just pulling together a list of popular courses and sorting them for the "Trending Courses" list.

Gross, I know. It meant our visitors would spend 3.5s staring at a white screen before anything rendered on-screen.

As a result, we spent a couple of days re-writing some of the more computationally demanding code. That brought load times down to ~4.5s, which was significantly faster. Tidying up the way we cached and retrieved results sped things up even more. How much more?

We're quite a bit faster today (2.6s) vs. two weeks ago (7.1s).

I should mention that I'm very proud of how our optimizations stood the ultimate load test, the Reddit hug of 500 concurrent visitors. After all, we're still running on the smallest 512MB droplet from DigitalOcean! No load balancers, no re-scaling, and no performance hiccups.

Aside: if you're looking to host a website on your own server, consider DigitalOcean through the referral link here. You'll be supporting us by covering the server bill for months, which itself is a testament to how inexpensive DO is!

Fewer bugs

No amount of optimization will fix a broken site. To that end, we've spotted out a handful of bugs (most have an insignificant impact) and squished them appropriately. The biggest bug we found, however, must have been very frustrating to some users. We apologize to those individuals.

Recently we made some changes that allow anyone to create and save lists. This makes it easy for someone who doesn't want to create an OpenCourser profile just to save the courses they're interested in.

A small subset of visitors will have noticed an error message on saving a new list. Those lists saved properly, but we redirected users to a broken endpoint, which resulted in those same visitors seeing an unhelpful "Internal Server Error" message. We've since made changes to make sure this doesn't happen again.

If you've created a list before, check the profile page to find it or send us an email at [email protected] with a brief description of your list.

Improvements + Fixes from Your Feedback

From the influx of visitors via Reddit came hundreds of comments, private messages, and feedback notes. We're very grateful for these.

In the past week, we've made several minor UI tweaks to streamline the experience based on your feedback. For example, we re-arranged the front page so that trending courses are listed vertically, rather than the odd looking three-column list.

We also cleaned up the individual course pages that many found overwhelming once you scrolled below the jump. That bottom half of the course page was very difficult to make sense of with so many stray links, buttons, text, and lists.

On the user experience front, we've made a small improvement to our save to list button across our site. Those buttons now rely on AJAX to send data to our servers, where previously they involved making a full browser request.

Before, we'd process that request and send back an empty response. Desktop and Android browsers process this response correctly, but iOS (iPhones and iPads) devices force a redirect, which made for some really jarring behavior. We ended up redirecting users to the same page they came from, but that was really a temporary workaround-- not a solution.

Sorry for the technical mumbo jumbo. What we're trying to say is iOS visitors will now have a better time on OpenCourser. If you haven't tried saving courses yet, now's the time to do it!

Searching Better

Finally, we received a lot of feedback along the lines of wanting suggestions for search results. The search bar is how most of our visitors find courses they enjoy and enroll in. Perhaps these changes will help you.

Aside from changes to performance, search has two new features:

The first is in response to one common question: "What if I don't have a subject I want to learn in mind?" To that, we built a little button that now suggests popular and/or unique search terms. If you're ever unsure of what to learn, tap or click it to fill your search bar with a suggestion. Go ahead, give it a spin!

The new suggestion button.

The second improvement we made to search let's visitors filter searches by course difficulty. Since launch, this has been the MOST frequently requested improvement. It's still in testing, but we're finally comfortable enough with it to release it as a beta. If you notice it acting up, let us know and we'll fix it.

The new difficulty filter comes in two flavors: beginner & advanced.

Secure Browsing

We care about your security, which is why we've taken some time to set up Let's Encrypt. When you use OpenCourser, you should now see the padlock in your browser that indicates you're on a secure connection.

While most data you send and receive through OpenCourser isn't particularly sensitive, we'd rather no one intercept your connection to us-- even if interception would mean something as benign as someone finding out what you're interested in learning about.

That's all for now. We <3 feedback. If you've got some for us, drop a comment below or email at [email protected].


Read more posts by this author.

Subscribe to Learn More

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!