The problem#

Slashdot recently made a change to their ad code that made the site completely fail to load for me, showing the message

Failed to load website properly since html-load.com is blocked. Please allow html-load.com

and then blaming ad blocking:

This page could not be loaded properly due to incorrect / bad filtering rule(s) of adblockers in use. Please disable all adblockers to continue using the website. (click OK if you'd like to learn more)

I could see the page loaded just fine behind those messages, so it was obviously a lie. But the page continually reloaded if I tried to dismiss those pop-ups, so the site was unusable.

(Actually, this appears to have been disabled in the time it took me to write this blog post, so I guess this is no longer needed. Although it may be applicable to other websites using the same or similar mechanisms.)

The solution#

Install this user script. I have only tested it with Greasemonkey on Firefox so it require modification to work on other user script managers or browsers.

The details#

The problem#

I was writing a user script where I wanted to be able to intercept the fetch() calls the web page made so my script could use the contents. I found a suggestion of simply reassigning window.fetch to my own function that internally called the real window.fetch while also doing whatever else I wanted. While it worked fine under Tampermonkey on Chromium, under Greasemonkey on Firefox, the script would just silently fail with no indication of why the code wasn't running.

(The script I was writing was this one for fixing the formatting on the Folklife 2024 schedule to reformat the schedule to display as a grid. I plan to write a devlog post on it in the future, but just writing about the most pernicious issue in this post.)

The solution#

The problem was that Firefox's security model special-cases function calls between web pages and extensions (and user scripts running inside Greasemonkey count as part of an extension for this purpose). And, furthermore, Promises can't pass the boundary, so you need to carefully define functions such that the implicit Promise created by declaring the function async lives on the right side of the boundary.

The following code combines all of that together to intercept fetch() on both Firefox and Chromium such that a userscript function intercept is called with the text of the response to every fetch():

const intercept = responseText => {
  // use responseText somehow ...
};

const w = window.wrappedJSObject;
if (w) {
  exportFunction(intercept, window,
                 { defineAs: "extIntercept" });
  w.eval("window.origFetch = window.fetch");

  w.eval(`window.fetch = ${async (...args) => {
    let [resource, config] = args;
    const response = await window.origFetch(resource,config);
    window.extIntercept(await response.clone().text())
    return response;
  }}`);
} else {
  const { fetch: origFetch } = window;

  window.fetch = async (...args) => {
    let [resource, config] = args;
    const response = await origFetch(resource, config);
    intercept(await response.clone().text());
    return response;
  };
}

The details#

The problem#

Previously, I showed how to get Firefox to show just the web content without any of the window borders or toolbars. But there's an obvious problem: those UI elements are actually useful for doing things with the browser. We can give a single URL as an argument when we start the browser, and that may be good enough for some use-cases, but what if we want to have more control over what the browser is displaying?

The solution#

Firefox has a feature for running automated tests called Marionette which we can use for automating Firefox outside of the context of running tests. There's an official Python client:

$ pip install marionette_driver
$ firefox --marionette &
$ python
>>> from marionette_driver.marionette import Marionette
>>> client = Marionette('localhost', port=2828)
>>> client.start_session()
{'browserName': 'firefox', ... }
>>> client.navigate('https://example.com/')

If it works, you should see Firefox load the URL https://example.com/.

You can find more information on the available commands on the basics page and the documentation.

The details#

The problem#

Web browser UIs have a lot more than just displaying the web page, which is useful when using them as a browser, but clutters the screen if all we want is to define what is displayed on part of the screen using HTML. So, can we get Firefox into a mode where it really does show just the website and nothing else? Firefox does have a fullscreen mode that does that, but it covers an entire monitor.

The solution#

To hide all of the Firefox menus and toolbars, put the following in the chrome/userChrome.css file under your Firefox profile directory (you will likely want to create a separate profile from the one you use for web browsing):

#TabsToolbar, #TabsToolbar-customization-target,
#nav-bar, #urlbar-container, #searchbar {
  visibility: collapse !important;
}

To hide the window border and titlebar, compile toggle-decorations.c and run

firefox &
./toggle-decorations $(xdotool selectwindow)

and then click on the Firefox window once it opens. It may be easier to bind it to a hotkey with xdotool getactivewindow or use some other way to identify the window.

The details#

The problem#

While the web developer tools in Firefox and Chrome provide a Storage/Application tab for inspecting the local data stored by a web app, neither shows OPFS files there, making it difficult to tell what's going wrong when you have a bug (which was a problem when writting my recent blog posts about OPFS). There's open Firefox and Chromium bugs about the missing feature, so if it's been a while since this was posted when you're reading this, hopefully this is no longer a problem.

Additionally, the tools I did find all use recursion, resulting in them failing to work on the deeply nested directory tree I created by accident.

The solution#

If you don't have several hundred levels deep of nested directories, you can just use this Chrome extension or this script (or probably this web component, although I couldn't get it to install), all named "opfs-explorer".

The following AsyncIterator returns all of the files in OPFS without using recursion and adds properties to include their full path and parent directory:

async function* getFilesNonRecursively(dir) {
  const stack = [[dir, "", undefined, 0]];
  while (stack.length) {
    const [current, prefix, parentDir] = stack.pop();
    current.relativePath = prefix + current.name;
    current.parentDir = parentDir;
    current.depth = depth;
    yield current;

    if (current.kind === "directory") {
      for await (const handle of current.values()) {
        stack.push([handle,
                    prefix + current.name + "/",
                    current,
                    depth + 1]);
      }
    }
  }
}

And here's the simple HTML display function I've been using that calls that (you will likely want to modify this to your preferences):

async function displayOPFSFileList() {
  const existing = document.getElementById("opfs-file-list");
  const l = document.createElement('ol');
  l.id = "opfs-file-list";
  if (existing) existing.replaceWith(l);
  else document.body.appendChild(l);

  const root = await navigator.storage.getDirectory();
  for await (const fileHandle
             of getFilesNonRecursively(root)) {
    const i = document.createElement("li");
    i.innerText = fileHandle.kind + ": "
                  + (fileHandle.relativePath ?? "(root)");
    if (fileHandle.kind === "file") {
      const content = await fileHandle.getFile();
      const contentStr = content.type.length === 0
                      || content.type.startsWith("text/")
        ? ("\"" + (await content.slice(0, 100).text()).trim()
          + "\"")
        : content.type;
      i.innerText += ": (" + content.size + " bytes) "
                     + contentStr;
    }
    l.appendChild(i);
  }
}

The details#