Ocf.io shorturls

Past discussion: ocf.io/rt/6638 (IRC chat search)

Currently we store useful shortlinks on ocf.io-- for example, ocf.io/docs points to documentation, ocf.io/servers points to our list of servers, and ocf.io/rt/### points to a numbered rt ticket. Right now, all of these are configured in a puppet config, which is annoying to edit. Also, since we use ocf.io shorturls to point to user websites (like ocf.io/dkessler), creating a shorturl requires adding that name to the list of reserved usernames, to prevent name collisions. This makes the process even more annoying.

We should redesign this system to make it easy for anyone to make a shorturl. This will require a lot of moving parts, and the design decisions have been controversial in the past-- see the discussion on this never-merged PR for some background.

Here are the necessary pieces for a good solution:

Use a separate namespace for some URLs

Put “temporary” shorturls in a different namespace so we don’t have to worry about most username collisions-- like ocf.io/s/temporary_url.

Database schema

Shortlinks will be stored in our MySQL database, instead of Git. We need to decide what kind of schema this will use, while taking into consideration both “static” links (like /servers) and “prepend” links (like /rt/###)

Webserver

We need to make a small application that responds to HTTP queries to ocf.io. It should read the URL, load the relevant shorturl from the database, and return an HTTP 302 redirect to the new page. As a bonus, we can implement caching on this server to reduce load on the database and speed up the response times (but honestly, we can probably do without this).

We also need to make sure this webserver is deployed cleanly in our infrastructure with Puppet.

UI

We can make UIs for creating short URLs in lots of places:

  • A web UI with text fields
  • A command-line tool
  • Add this functionality to create, our IRC chat bot

Username collision check

Even though we will use a separate namespace for “temporary” shorturls, we should still add code to our username validator to ensure that new accounts aren’t created with names that collide with permanent shorturls.

we can avoid 100% of the username collision problem for transient shorturls by using a shorturl namespace like ocf.io/s/

for permanent shorturls like ocf.io/join or ocf.io/password, we take care of this manually by blacklisting the usernames in ocflib.

1 Like

I can’t imagine any shorturls realistically having a collision with someone’s username though. Most shorturl names will be something like “signup-{event}” or “{event}-slides”.

Further, I don’t think it actually matters to have a distinction between “permanent” and “temporary” shorturls. Why not put everything in the same namespace? It’s not like the “permanent” URLs carry any guarantee of immutability-- maybe, for example, we’d move ocf.io/password to a different URL.

Certain “permanent” shorturls should be immutable, because we’ve already widely distributed them. I wouldn’t change many of them without redirects, at the very least.

Namespaces are also a nice way to organize shorturls. Prefixing shorturls with “signup” or “slides” is already an implicit namespace, so I guess we don’t really disagree on that point.

Having something like ‘ocf.io/s/’ as a shorturl-project namespace lets us continue to use the existing puppet code for shorturls, which works great for things like ocf.io/rt/, ocf.io/gh/, which want just a little bit of smart regex matching, and let us simplify the implementation of a database-backed shorturl engine to simply “select url from shorturls where slug = `” and ship it to users. No regex parsing necessary, as we can let Apache do that.

1 Like

I also agree with distinguishing permanent and temporary shorturls - we shouldn’t be careless about what parts of the username namespace we lop off.

I’m +1 on abizer’s proposal as well. As a bonus, it’ll reduce complexity of the implementation, albeit slightly.

OK, I guess the people have spoken on this one. We can use a separate namespace for these, then.

In any case, we should still eliminate the requirement to edit two separate files in order to add a “permanent” shortlink.

Perhaps the best solution really is then to make all of ocf.io a webapp for serving short URLs. We can have an s namespace backed by a database, and put non-s shortlinks in ocflib.

1 Like

I got a working solution using yourls.

for example,

the page where a user can add stuff is located at /s/adminocf.io/minos/s/admin
there is a plugin for ldap integration which we can use to allow staffers to login. also we can ocf label the admin webpage down the road.

nice thing about this solution is that it fulfills all the requirements that were brought up in dkessler’s original post.

I think @awelty wrote his own Flask app for this a few months ago as well. I’m willing to evaluate using externally written software as well but I’m always gonna be a little distrustful of PHP. Also for something this simple it might be better for us to roll our own application.

I would like to see this getting done soon, this project has been in a limbo for awhile – that’s main drive of using Yourls solution and get it implemented.

github issues on Yourls – there are tones of issues, security issues and what not with this simple idea of shorturl redirection. do we really want to reinvent the wheel and run into those problems down the road? that also adds the component that we have to maintain the software we write. Yourls seems to be actively developed, so just maintaining the version might be easier for us… like how we do with phpmyadmin.