10% of the 666 most popular Python GitHub repos have these f-string bugs


We found that 10% of the 666 most popular Python open source GitHub repositories had the following f-string typo bugs:

  • f prefix was missing: "hello {name}" instead of f"hello {name}" .
  • f prefix was written inside the string: "fhello {name}" instead of f"hello {name}" . We even saw an example of "hello f{name}" .
  • Concatenated strings used the f prefix on only the first string: f"hello {name}" + "do you like {food}?" instead of f"hello {name}" + f"do you like {food}? .

We fixed the problems we found in 69 of the most popular open source Python repositories — repositories you probably know and might even use. At the bottom of this page is a list of the GitHub pull requests we created but for the sake of summary some were Tensorflow, Celery, Ansible, DALI, Salt, Matplotlib, Numpy, Plotly, Pandas, Black, readthedocs.org, Scipy, PyTorch, rich, and some repositories from Microsoft, Robinhood, Mozilla, and AWS.

Why not follow me on Twitter to if you like this type of thing.

What is f-string interpolation?

Literal String Interpolation aka f-strings (because of the leading character preceding the string literal) were introduced by PEP 498. String interpolation is meant to be simpler with f-strings. “meant” is foreshadowing.

Prefix a string with the letter “ f ” and it’s now an f-string:

name = "jeff"
print(f"My name is {name}")

The mistakes/bugs/issues

We checked 666 most popular Python open source repositories on GitHub and found 10% of them had three types of f-string bugs, all of which broke the formatting leading to the value of the interpolated variable not being included in the string when evaluated at runtime.

Mistake 1: Missing f prefix

This was the most common case we found, and occurred mostly during logging and raising exceptions. This makes sense because logging and exception raising is a common use case for printing variable values into string messages. Take this example from pymc:

for b in batches:
if not isinstance(b, Minibatch):
raise TypeError("{b} is not a Minibatch")

Some developer debugging that TypeError is not going to have a great time as they will not immediately know what b equals due to the missing f prefix.

Mistake 2: accidentally including f inside the string

This is probably the hardest for a code reviewer to spot because if we give the code a cursory look before quickly typing “LGTM!” we can see there is an f at the start of the string. Take this example from Lean, would you spot it the f in the wrong place?

RateOfChangePercent('f{symbol}.DailyROCP({1})', 1)

Or this problem in Keras?

raise ValueError('f{method_name} `inputs` cannot be None or empty.')

Or this one from Buildbot


Many developers would not notice this during code review because this is a type of error humans are bad at spotting. Humans often see what they expect to see, not what is really there. That’s what makes static analysis checkers so important, especially ones that can also suggest the fix like so:

This highlights that for manual processes like code review to detect 100% of bugs 100% of the time then 100% of humans must perform 100% perfectly during code review 100% of the time. See the cognitive dissonance? We do code review because we expect human error when writing the code, but then expect no human error when reviewing the code. This dissonance results in bugs being committed.

Mistake 3: f-string concatenation

During concatenation only the first string is prefixed with f . In this example from Tensorflow the second string in the concatenation is missing the f prefix:

raise ValueError(
f"Arguments critical_section_def={critical_section_def} "
"and shared_name={shared_name} are mutually exclusive. "
"Please only specify one of them."

This type of f-string bug is perhaps the most interesting because a pattern emerges when we look at all of the example of this issue:

  1. The first string in the concatenation is correctly prefixed with f .
  2. The second string is not prefixed with f.

We may be looking too deep into this but it seems like many developers think when string concatenation occurs it’s enough to declare the first string as an f-string and the other strings are turned into f-strings by osmosis. It doesn’t. We’re not suggesting this is the case for all developers that accidentally did this error, but interesting nonetheless.

The impact of f-string bugs

Based on the 666 repositories we checked, broken string interpolation due to typos most commonly occurs during logging, exception raising, and during unit testing. This therefore impacts confidence in the proper functioning of the code, and also impacts developer efficiencies when debugging.

f-string mistakes during logging

Bad string interpolation during logging means the information the code author intended to express will not actually be available to the developer trying to debug the code. For example see this code taken from Celery:

self.time_to_live_seconds = int(ttl)
except ValueError as e:
logger.error('TTL must be a number; got "{ttl}"')

The developer will look in the logs and see got "{ttl}" and not see the value of ttl. It’s ironic that a developer will read the logs to debug something, only find another unrelated f-string bug that hinders them from fixing the original bug they were investigating.

Note also that Python docs suggest not using string formatting during logger calls. Instead use the logger’s build-in-mechanism to benefit lazy string interpolation for runtime optimisation.

f-string mistakes during exception raising

The impact is similar to the impact of bad f-string declaration during logging: it will reduce the effectiveness of a developer during debugging. Take for example this exception handling code from Microsoft CNTK:

raise ValueError(
'invalid permutation element: elements must be from {-len(perm), ..., len(perm)-1}'

Or this example from Tensorflow:

raise ValueError(
"{FLAGS.tfrecord2idx_script} does not lead to valid tfrecord2idx script."

Both of those should be f strings but are missing the f prefix.

The good news is crash reporting tools such as Sentry or Crashlyrics should show the variable value during the crash. Still makes debugging harder though, and not all projects use such tools.

f-string mistakes during unit testing

Tests give confidence that our code behaves how we want. However, sometimes it’s the tests themselves that don’t always behave how we want. Perhaps due to typos. In fact perhaps due to f-string typos. Take for example this test found in Home Assistant:

response = await client.get(
assert response.status == HTTPStatus.NOT_FOUND

The test expects a 404 to be returned due to “invalid-event-id” being used in the URL but actually the cause of the 404 is for another reason: because the device.id value was not interpolated into the string due to a missing the f prefix. The test is not really testing what the developer intended to test.

There is some poor developer out there maintaining the this part of the codebases that is relying on this test and using it as a quality gate to give them the confidence they have not broken the product but the test is not really working. It’s this kind of thing that can harm a night’s sleep! It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.

Detecting the bugs

Code Review Doctor is a static analysis tool that detects common mistakes and offers the fix. When adding new checkers we stress test them by analysing open source code, and then check the results to minimise false positives.

To test our new “this probably should be an f string” checker we generated a list of the most popular Python repositories on GitHub by using GitHub’s topic search API. We ran the following code (some code is skipped for simplicity):

class Client:
    def get_popular_repositories(self, topic, count, per_page=100):
        urls = []
        q = f"topic:{topic}&order=desc&sort=stars"
        url = f'https://api.github.com/search/repositories?q=q'
        first_resp = self.session.get(
            params={"per_page": per_page}
        for resp in self.paginate(first_resp):
            urls += [i['clone_url'] for i in resp.json()['items']]
            if len(urls) >= count:
        return urls[:count]
    def paginate(self, response):
while True:
yield response
if 'next' not in response.links:
response = self.session.get(

As you can see, GitHub uses the Link header for pagination. We then ran that code like:

client = Client()
clone_urls = client.get_popular_repositories(
topic='python', count=666

Here’s the gist of the list of GitHub repository clone URLs that was generated.

Once we got the list of most popular Python repositories we ran them through our distributed static analysis checkers on AWS lambda. This serverless approach allows us to simultaneously test hundreds of repositories with perhaps millions of lines of code without having to worry about scalability of our back-end infrastructure, and we do not require our developers to have beefy development machines, and our servers don’t go down or experience performance degradation during peak load.

Minimising false positives

False positives are annoying for developers, so we attempt to minimise false positives. We were able to do this by writing a “version 1” of the checker and running it against a small cohort of open source repositories and check for false positives, then iterate by fixing the false positives and each time increase the number of repositories in the cohort. Eventually we get to “version n” and were happy the false positives are at an acceptable rate. Version 1 was very simple:

GIVEN a string does not have an f prefix
WHEN the string contains {foo}
AND foo is in scope
THEN it’s probably missing an f prefix

This very naive approach allowed us to run the checkers and see what cases we did not consider. There were a lot of them! Here are some false positives we found:

  • The string is later used in an str.format(…) call or str.format_map(…)
  • The string is used in a logger call. Python logger uses it’s own mechanism for string interpolation: the developer can pass in arbitrary keyword arguments that the logger will later interpolate:logger.debug("the value of {name} is {value}. a={a}.", name=name, value=value, a=a)"
  • The string is used in behave style test @when('{user} accesses {url}.')
  • The “interpolation” is actually this common pattern for inserting a value into a template some_string_template.replace('{name}', name) .

The false positive that was most difficult to solve was the string later being using with str.format(…) or str.format_map(…) because there is no guarantee that the str.format(…) call occurs in the same scope that the string was defined in: the string may be returned by function to be later used with str.format(…). Our technology currently does not have the capability to follow variables around every scope and know how it’s used. This will be an interesting problem to solve.

With these false positives in mind, our BDD-style spec ends up like:

GIVEN a string does not have an f prefix
AND the string is not used with an str.format call
AND the string is not used with an str.format_map call
AND the string is not using logger’s interpolation mechanism
AND the string is not used in @given @when @then
And the string is not used in .replace(‘{foo}’, bar)
WHEN the string contains {foo}
AND foo is in scope
THEN it’s probably missing an f prefix

That is vast simplification: there are many other ways a string can “look” like it should be an f-string but actually is not.

Reaction from the community

It was interesting to write the f-string checker, it was interesting to detect how common the problem was (10% is a lot higher than we expected!), and it was interesting to auto-create GitHub pull requests to fix the issues we found.

It was also interesting to see the reaction from open source developers to unsolicited pull requests from what looks like a bot (really a bot found the problem and made the PR, but really a human developer at Code Review Doctor did triage the issue before the PR was raised). After creating 69 pull requests the reaction ranged from:

  • Annoyance that a bot with no context on their codebase was raising pull requests. A few accepted the bugs were simple enough for a bot to fix and merged the pull request, but a few closed the pull requests and issues without fixing. Fair enough. Open source developers are busy people and are not being paid to interact with what they think it just a bot. We’re not entitled to their limited time.
  • Neutral silence. They just merged the PR.
  • Gratitude. “thanks” or “good bot”.
  • A developer even reached out and asked us to auto-fix their use of using == to compare True False and None, so we did. Read more about that here.

For science you can see all the reactions (both positive and negative) here.

Protecting your codebase

You can add Code Review Doctor to GitHub so fixes for problems like these are suggested right inside your PR or scan your entire codebase for free online.

Here’s a list of pull requests we created:

Pull Request

3% of 666 Python codebases we checked had silently failing unit tests (and we fixed them all, including PyTorch, Salt, Python.org)

Lets coin a name for a very special type of unit test:

Schrödinger’s unit-test: a unit test that passes but fails to test the thing we want to test.

This test will not fail

This article focuses on our code scanning of 666 Python codebases to detect one such Schrödinger’s unit tests. Specifically, this gotcha (which we found in 20 codebases and later pushed an auto-generated fix for):

Do you see the mistake? It’s a relatively common mistake for a developer to make. assertEqual and assertTrue were muddled up. In fact of the 666 codebases we checked, we found 3% of them mixed the two up. This is a real problem that affects real codebases, even active large ones with a huge community and strong testing and code review practices:

Python.org (ironically enough)
UK Ministry of Justice
Django CMS

See the full lists in this gist. The fact the problem is so widespread suggests a key cause of this mistake is because it’s so easy to miss during code review. A quick glance at the code does not raise any red flags. It “Looks Good To Me!”.

Schrödinger’s unit-test

In the late 19th century Mark Twain apparently said It’s not what you know that gets you. it’s what you think you know but ain’t. What was true in late 19th century literature is also true in early 21st Python unit testing: tests that don’t really test the thing we want to test is worse than no tests at all because a Schrödinger’s test gives unwarranted confidence that the feature is safe.

Have you ever encountered this workflow:

  • Code change committed, and the tests for the feature passes locally.
  • The test ran in CI during the pull request build, and no failures reported.
  • The change was peer reviewed and accepted during code review. (“LGTM!”)
  • The change was then deployed to prod… only for it to fail upon first encounter with the user
  • After reading the bug report proclaim to yourself “but the test passed!”

Upon revisiting the unit test it’s then noted that the unit test that gave such great confidence during the software development life cycle was in fact not testing the feature at all. If you encountered this problem then you encountered this class of unit tests which this specific TestCase.assertTrue is one of.

The TestCase.assertTrue gotcha

While pytest is very popular, 28% of codebases still use the built-in unittest package. The codebases that do use the built-in unittest package are at risk of the assertTrue gotcha:

The docs for assertTrue state that test that expr is True, but this is an incomplete explanation. Really it tests that the expr is truthy, meaning if any of the following values are used in as the first argument the test will pass:

'foo', ‘bar’, 1, [1], {‘a’}, True

And conversely with falsey values the test will fail:

'', 0, [], {}, False,

assertTrue also accepts a second argument, which is the custom error message to show if the first argument is not truthy. This call signature allows the mistake to be made and the test to pass and therefore possibly fail silently.

For example when making 20 PRs fixing the problems we found one of the tests started to fail due to faulty application logic once the test was no longer a Schrödinger’s unit test:

We also found that at least two of the tests failed because the logic in the test was wrong. This is also bad if we consider unit tests to be a form of documentation describing how the product works. From a developer’s point of view, unit tests can be considered more trustworthy that comments and doc strings because comments and doc strings are claims written (perhaps long ago and perhaps now obsolete or incomplete), while a passing unit test shows a logic contract is being upheld…as long as the test is not a Schrödinger’s test!

Perhaps more data is needed, but this could mean 15% (3 in 20) of Schrödinger’s unit tests are hiding broken functionality.


When the CodeReview.doctor github bot detects this mistake in a GitHub pull request the following solution is suggested:

You can securely scan your entire codebase online for free at CodeReview.doctor

Hacking Django websites: clickjacking

Part 2 of the “Hacking Django” series: part 1, part 3

Clickjacking is an attack where one of your logged-in user visits a malicious website, and that website tricks the user into interacting with your website via an iframe.

As an example, see the green “create pull request” button which will create a Pull Request on GitHub as the logged in user:

Let’s say the malicious website has some nefarious React:

import React from 'react';

export default function (props) {
  const [position, setPosition] = React.useState({ clientX: 0, clientY: 0 });
  const updatePosition = event => {
    const { pageX, pageY, clientX, clientY } = event;
    setPosition({ clientX, clientY,});

  React.useEffect(() => {
    document.addEventListener("mousemove", updatePosition, false);
    document.addEventListener("mouseenter", updatePosition, false);
    return () => {
      document.removeEventListener("mousemove", updatePosition);
      document.removeEventListener("mouseenter", updatePosition);
  }, []);

  return (
        style={{zIndex: 100, position: 'absolute', top: position.clientY-200, left: position.clientX-650}}
      <div>Some website content</div>

When one of the logged in users accesses the malicious website this happens: the iframe follows the mouse so that the button is on the mouse. When the user clicks, they click on the iframe.

Now that is very obvious – the user can see the iframe, but that’s one change away: style={{display: 'none', ...}}. For the sake of the demo I used style={{opacity: '0.1', ...}}(otherwise you would see nothing interesting):

Clickjack prevention middleware

The solution is simple: to set iframe embed policy for your website: adding django.middleware.clickjacking.XFrameOptionsMiddleware to settings MIDDLEWARE and X_FRAME_OPTIONS = 'SAMEORIGIN' will result in X-Frame-Options header with value of SAMEORIGIN, and modern browsers will then prevent your website from being embedded on other websites.

Continue reading: part 1, part 3

Does your website have security vulnerabilities?

Over time it’s easy for security vulnerabilities and tech debt to slip into your codebase. I can check clickjacking vulnerability and many others for for free you at https://django.doctor. I’m a Django code improvement bot:

If you would prefer security holes not make it into your codebase, I also review pull requests:


{% static %} handles more than you think

You probably agree using {% static %} is not a controversial:

Credit: Django Doctor

If you’re unfamiliar: {% static %} returns the path the browser can use to request the file. At it’s simplest, that would return a path that looks up the file on your local file system. That’s fine for local dev but in prod we will likely use third-party libraries such as whitenoise or django-storages to improve performance of the production web server.

There are reasons other than “because S3” for why we use {% static %}

File revving

Whitenoise has a storage backend STATICFILES_STORAGE that performs file revving for cache busting. As a result the file path rendered in the template is renamed: script.js may be renamed to script-23ewd.js. The STATICFILES_STORAGE generates a hash from the file’s contents and renames the file and generates a manifest file that looks like:

{ 'scripts.js': 'scripts-23ewd.js'}

This is called revving. {% static ‘script.js’ %} renders 'script-23ewd.js' in the template because 'script.js' is looked up in the manifest, resulting in 'script-23ewd.js' being resolved.

This is for cache busting: when the file content change the file name changes too, so a revved file can have cache headers set to cache forever in the CDN or browser. On the next deployment if the file contents changes so too does the file name being requested and thus the most up to date file is retrieved by the browser.

So {% static %} abstracts away cache-busting mechanisms so we can focus on templatey things in the template instead.

Websites can move house

Given enough time a website can change where it’s served from:

  • From subdomain to path: abc.example.com to example.com/abc/ and xyz.example.com to example.com/xyz/. This has happened in a project I was in for SEO reasons.
  • Merging multiple repositories into one. This happened to a few projects I was involved with to make development more efficient.

At this point it would be unclear which app’s files are served from /static/. Yes a developer can find a replace all of /static/ with /abc/static/ and /abc/static/. But if {% static .. %} was used they would not need to.

So {% static %} prevents Javascript and CSS from breaking because where the website is served from changed.

Serving from S3

Django suggest not serving static files from the web server in production. One way to do that is to serve the files from S3. If STATICFILES_STORAGE serves from S3, then the static domain will likely be different from webserver domain unless some configuration in Cloudfront ensures otherwise.

So in that case using {% static ‘script.js’ %} ultimately renders https://s3-example-url.com/script.js instead of /static/script.js.

So {% static %} handles changes in the domain the files are served from.

Do you {% static %} everywhere you should?

You can check if your codebase at django.doctor. It check missing {% static %} and other Django code smells:

If you want to avoid code smells getting into your codebase in the first place install the GitHub PR bot to reduce dev effort and improve your code.

Alexa audio effects: change Alexa’s voice

Ever wished Alexa sounded like she was an old-timey radio? Sports stadium announcer Alexa? Alexa behind a door? Alexa Browser Client has you covered.


Run the demo then go to

Why? Because it’s cool. But also to demonstrate the advantages of running Alexa in the browser: we have full control over the audio received from Alexa Voice Service. Adding audio manipulation takes not too many lines of code.

Old-timey radio Alexa

Sports stadium Alexa

Alexa Brower Client – Talk to Alexa from your desktop, phone, or tablet browser.

I made a pluggable Django app that allows you to talk to Alexa through your browser:

  1. Add ​​​​​alexa_browser_client to your INSTALLED_APPS
  2. Run the Django app ./manage.py runserver
  3. Go to http://localhost:8000/alexa-browser-client/
  4. Say something interesting to Alexa e.g., “Alexa, What time is it?”



And hear Alexa’s answer through your browser,

Buy why?

  1. It’s like the Amazon Echo, but web developers can build a rich web app around it – and that web app can be used on your Desktop, Tablet, Smartphone, or Smart Watch.
  2. You can change the wakeword to anything you want. e.g., “Jarvis“.
  3. You can be 100% certain that Amazon, CIA, or “Russian Hackers” are not always listening: the Alexa code does not fire up until the “Alexa” wakeword is issued.
  4. Voice control is an important technology, and will only become more relevant. 11 Million households now have an Alexa device, 40% of adults now use voice search once per dayCortana now has 133 million monthly users. If you’ve not started yet, it’s worth getting into voice technology to stay relevant.

So Alexa Browser App is a tool for developers to build voice commands into their web apps. The only limit is yourself.

Next Steps

I will be using Alexa Browser Client to build a rich web app for controlling my smart home devices. I will put cheap Android tablets in each room, and the web-app will be loaded on each tablet.

Has someone knocked on the door? “Alexa, ask the house to video chat with the front door”

Want to view your Google Calendar? “Alexa, ask the house to show my calendar”.

Want to view the traffic situation in Google Maps? “Alexa, ask the house to show my commute”.

80% of being married is shouting “What did you say?” to your spouse in another room. No more: “Alexa, ask to the house to video chat with the kitchen”.

Note in the above examples you would need to create a custom Alexa Skill that reacts to “the house” and routes the command to your smart home controller app.


Alexa Browser Client has great test coverage, highest Codeclimate score, and Gemnasium integration – but the library does not yet support all the Alexa Voice Service features. Feel free to make a Pull Request to add features. As a community, we can make a fully-featured customizable open-source Echo alternative in the browser.

When four equals five

Consider a form that allows users to search for holidays. It is expected behaviour that the form remembers previous searches –  so if a user previously searched for “4 guests”, the form should pre-populate with “4 guests” the next time the page loads.

This is a simple enough idea, but an odd bug was raised for this component. Ten points to Gryffindor for spotting it before knowing the symptoms: click to see the code.

You read the code code, yes? And you spotted the bug? No? Either way a summary:

1) The components are split into ‘container’ and ‘presentational’. Here is why.
2) createContainer returns a component instance that had state from a flux store passed to it as props.
3) If state.enquiryForm.numberOfGuests is falsey then default to 1.
4) We have some tests that confirm logic is as expected (1 for falsey, 5 for otherwise).

The unit tests passed, and we were happy when we manually tested the component in the browser — the select box ‘remembered’ the value. Great. Deploy!

So why was there a bug? The report was “the number of guests select box adds 1!”

After checking the component and the code we concluded ‘nope’ — The component did not add one to the value — when we selected “one guest”, on reload we got “one guest” in the form. When we selected “five guests” we got “five guests” on reload. When we select “four guests” in then we get “five guests” on reload. Wait, what?

Investigating the bug

Yes, the bug was reproducible . Great. Below is a table showing ‘the value we selected in the form’ against ‘the value pre-populated on refresh’:

Screen Shot 2016-05-22 at 22.40.53
Every even number adds one to itself!? Okay…
Go back to the code and see if you can see the bug. No?

Bitwise OR, Logical OR

There’s a big difference between | and ||. Due to a simple typo a bitwise OR was used in lieu of a logical OR. The outcome of the missing pipe was this unexpected behavior.

A bitwise OR performs a logical OR on each bit of in the numeric value — so five will be converted to 0101, and one will be converted to 0001, and each bit will be OR’d — if either of the bits are 1 then use 1 in the return value’s position:


But when we use an even number such as four or one, we get:


and again with six or one:


but to labour the point with seven and one:


Lessons learned

‘Write more unit tests’ is not the answer to this problem. After all, unit tests were written and the bug still got through. We could write tests specifically targeting ‘bitwise OR’ bugs, but this sets precedence against all similar scenarios. This would take an inordinate amount of time. Moreover, unit tests are for confirming logic, not detecting bugs. This type of bug can be more effectively detected using other means.

This bug was missed by three developers during code review. Does that indicate a human problem with the review process? I think not. Sometimes humans see what they expect to see. With this in mind, as put by W Edwards Deming: 100% inspection is only 100% effective if the inspector detects 100% of defects 100% of the time…which is impossible due to human nature. After all, if humans were that precise there wouldn’t be any defects made in the first place.

We can make ourselves more precise though by using static analysis tools like eslint. We turned on no bitwise-operators rule and it shows a helpful warning message. It should not interfere with the day to day tasks  of the project— if an e-commerce site sees itself manually wrangling bits then it’s probably doing something it shouldn’t.

Bloom filter as a service: validate username availability without asking the server.

Consider a spreadsheet user management interface that allows creating and updating users in web spreadsheet. It must validate that usernames have not been used before:

Screenshot from 2015-09-02 22:39:20

An obvious solution is to ask an endpoint if the username has already been taken. However, the admin user will experience a small lag due to the round trip from client to server: there will be at least a tenth of a second lag before the user is informed the username cannot be used. The impact is compounded when the admin user is managing many users in the spreadsheet interface.

This post will explain how to avoid that round trip completely and thereby eliminating the delay between when the user enters text and when they are prompted it’s invalid, which improves user experience.

We want the browser to have knowledge of all taken usernames without knowing what those usernames are, and without needing to call the server.

We obviously won’t give the browser a list of usernames. Words like “hacker” and “brute force attack” spring to mind. Some of those usernames will have really weak passwords! Plus there are privacy concerns because some of those usernames might be email addresses that can identify an individual human.

So, how can the browser have this knowledge without having the usernames?

Bloom filters

Skip this section if you already know what a Bloom filter is.

A Bloom filter is a space efficient data structure for testing if an element is a member of a set. Once the element (e.g., the string “How it’s Made”) is added to the Bloom filter it can later tell you whether it has already seen “How it’s Made” without having to store the literal value – which saves no end of space when we consider that millions of elements can be added to the Bloom Filter.

A simple interface might look like:

> bloomFilter.add("Get your own back")
> bloomFilter.check("Get your own back")
= true
> bloomFilter.check("not found")
= false

Internally the Bloom filter is a bit array. When we add “50/50”, some probabilistic hash functions convert the value to a series of integers. These integers represent indexes of the bit array, and the bits at these indexes are switched to 1.

Later when we check if “50/50” has been seen, the value is passed through the same hash functions returning the same series of indexes. If all bits at those indexes are 1 then it is assumed the value has been seen before. There is risk of false positives: the Bloom filter might erroneously say something has already been encountered.

For hands on, see this interactive example.

Bloom filter for profit

So, by now you see we can add all the used usernames to a server-side Bloom filter, but it might look like we’re no closer to out goal – the bloom filter would still be on the server, and we would still need to send the username to it to check validity. Well, straw man, that’s incorrect!

We can expose the Bloom filter’s internal bit array to the browser so the browser can use it to create a new Bloom filter on the browser, without any further need to talk to the server! The general flow:

  1. Create a micro service that maintains a Bloom filter – adding usernames to the Bloom filter.
  2. Expose the component pieces of the Bloom filter on an endpoint, so the Bloom filter can be re-created elsewhere.
  3. A browser calls the endpoint and creates the client side bloom filter, then uses the client-side bloom filter for validating usernames.

I created a super simple proof-of-concept bloom filter as a service using Node and ExpressJS – which is available on github. I’m a lazy namer so I called it “bloomy”.

Client side

I implemented the client-side code in an Angular app. It looks like this:

// `getBloomy` promise resolves with response from bloomy's endpoint. Note `BloomFilter` is an global object introduced by including the file `public/javascripts/BloomFilter.js`, which is included in the github repo.
getBloomy().then(function(response) {
    myService.bloomFilter = new BloomFilter({capacity: response.capacity});
    myService.bloomFilter.filter = response.filter;
    myService.bloomFilter.salts = response.salts;
// later on, using the client side bloom filter
myService.bloomFilter.check(value) // true or false

The `getBloomy` code is called in the `resolve` property of angular-ui-router FWIW.

Now we have eliminated the need to call the server when the user enters the username – the browser already “knows” if the username has not been taken. Spooky!

There are lots of room for improvement, and my use-case and tech stack will likely be different to yours, but that’s why forking and PR’s were invented 🙂

The solution is not without its problems, however:

  • The server-side and client-side Bloom filter implementations must be exactly the same for exporting and importing of bit arrays to work. I tried making a Python Bloom filter interoperable with Javascript. Take heed it creates only headaches. The easiest thing is to use exactly the same Bloom filter implementation on client and server side – which I have included in the repo.
  • Hackers can extract usernames from the client-side Bloom filter by brute forcing the Bloom filter (“bloomFilter.check(‘a’); bloomFilter.check(‘aa’); bloomFilter.check(‘aaa’)). Brute force attacks on you login endpoint can be rate limited. Not if the Bloom filter is on the client side! This might be solvable with further investigation.
  • Data consistency problems/single source of truth issues: a user can be created on the server after the point the server-side bloom filter was exported. The client-side bloom filters will now be unaware a username is no longer available, and erroneously say the username is valid. This can be worked around by pushing the bloom filter bit array to browsers whenever the bloom filter changes. This can get mad hairy.
  • Once an item is added to the Bloom filter it cannot be removed. This is possibly OK for usernames – as there is a schools of thought that argues usernames should be historically unique. However, if we want to remove items, we can implement a Cuckoo Filter instead.
  • Currently the repo code assumes a max Bloom filter size of 10,000. You might have more. This can be resolved by using a scalable bloom filter.


The “bloom filter as a service” approach replaces 1 tiny user experience problem with a few technical problems which if mismanaged can lead to huge user experience problems such as incorrect validation results.

Moreover, if the use-case is username validation then an attack vector is introduced. Let’s not do that in production. A better usage in real life could be any other client-side validations that involve testing against a whitelist or blacklist such as “adding a tags to a post”, “preventing the user inserting links to naught websites”, etc.

Angular Material Design custom build: include only the parts you need

I reduced the file size of my app’s dependencies by 15% by only including the Angular Material source code my app actually uses. This simple action improved page load speed and this should reduce bounce rate. As the app in question is another app’s marketing app/landing page, this improvement can have real monetary value.

The landing page is built with Angular Material (because so is the app it advertises). Angular Material is heading for feature parity with the components detailed in the Material Design spec. These features are awesome, but the landing page doesn’t use them all.

For the landing page, these unused features are wasteful: user wait for bloat to download that the app never uses. If we strip away unused features from our app then we can make the app load a little bit faster.

This post will explain how to easily make a custom Angular Material build using Gulp that includes only the needed features, without having to pull the Angular Material repo.

Angular Material Design “modules”

The Angular Material bower package comes with a “/modules/js/” folder, which has each of the components split into different source folders:


This makes it super easy for us to build only the components we need. We will need to do a bit of boilerplate to define the angular module that these components will live under.


My build tasks looks something like below. When “buildDependencies” task is ran, it first finishes “buildMaterial” task, which creates “custom.js” in the angular material folder. “buildDependencies” task  then picks up custom.js and concatenate it with the other dependencies.

var insert = require('gulp-insert');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var DIR_JS_LIBS = './assets/js/libs';
var DIR_BUILD = './build/'
gulp.task('buildMaterial', function () {
    var modules = [
    var boilerplate = [
        '(function(){' +
        '    angular.module("ngMaterial", ["ng", "ngAnimate", "ngAria", "material.core", "material.core.theming.palette", "material.core.theming", "material.components.toast"]);' +
        '    })();'
    return gulp.src(modules, {cwd: DIR_JS_LIBS})
        .pipe(concat({path: 'custom.js', cwd: ''}))
        .pipe(gulp.dest(DIR_JS_LIBS + 'angular-material/'));

gulp.task('buildDependencies', ['buildMaterial'], function () {
    var deps = [
    return gulp.src(deps, {cwd: DIR_JS_LIBS})
        .pipe(concat({path: 'deps.min.js', cwd: ''}))


The above example is a stripped down version of the real task. In reality the project uses about 7 components. After minifying I saw I saved about 60kb by using the custom build. This is approximately a 15% reduction.

This is not a huge saving, but it is worth the effort if continual improvement philosophy is followed then many small improvements can lead to great savings.

FWIW, click here to see my project using the custom Angular Material build. Its a marketing app for “Monofox Ask Peers” – basically a slimmed down private stack overflow for schools.

Memoize AJAX requests without data inconsistency

A common problem working with ajax is firing duplicate requests. A common solution is to first check if a request is ongoing. I solved this problem differently by memoizing the get request for the lifetime of the request.

Memoization is storing the results of a function call and returning the cached result when the same inputs occur. This is often used to optimize expensive function calls, but I see value in using it here. In the code below we use Restangular, but the concept works just as well for any promise based request library.

function memoizedService(route) {
    var service = Restangular.service(route);
    var hasher = JSON.stringify;
    var memoizedGetList = _.memoize(service.getList, hasher);
    * Get list, but if there already is a request ongoing with the same
    * params then return the existing request.
    * @param {Object} params
    * @return {promise}

    service.getList = function(params) {
        return memoizedGetList(params).finally(function() {
            delete memoized.cache[hasher(params)];

     return service;
var questionEndpoint = Restangular.memoizedService('question');

Make sure you install underscore with a bit of

bower install underscore --save

Then, to use the feature we would do:

questionEndpoint.getList({page: 1});

If we request page 1 when there is already a request for page 1 then the second `getList` will return the same promise that the first request returned and you will only see 1 request for page 1 in the network panel. Note importantly we also remove the cached request when the request is complete. This prevents data consistency problems: a get request can return different data over time (e.g., more records), and we want to make sure the user receives the most up to date data.