Hugo Sum

How do I search and replace text in Neovim

Sat, 30 Aug 2025 00:00:00 GMT

Searching and replacing text is something a developer would do frequently. Unlike other editors such as Visual Studio Code, Neovim doesn’t come with a search box for you to search and replace text in a file or in a workspace, and it has it own unique mechanism. In this post I want to share how do I search and replace text efficiently in Neovim.

Searching and replacing text in a buffer with `:substitute` command

Searching and replacing text in Neovim is a bit different from what you might be used to in other editors. Instead of a search box, you use the powerful :substitute command, which gives you fine-grained control over your replacements.

To replace all occurrences of foo with bar in the entire file, you can use:

:%s/foo/bar/g

:% means the whole file.
s stands for substitute.
/foo/bar/ is the pattern and replacement.
g means “global”—replace all matches on each line.

If you only want to replace in a specific range, say lines 10 to 20:

:10,20s/foo/bar/g

Sometimes, you want to confirm each replacement before it happens. Add the c flag:

:%s/foo/bar/gc

Neovim will prompt you for each match, letting you decide whether to replace it.

You can also ignore cases by adding the i flag:

:%s/foo/bar/gci

For a deeper dive into the :substitute command, the official Neovim documentation is a great resource.

Searching and replacing text in a buffer with `gn` motion

While the :substitute command is powerful, sometimes you want a more interactive way to search and replace text—especially when you only want to change some matches, not all. That’s where the gn motion comes in.

The workflow is simple:

Search for the text you want to replace using /pattern or *.
Use n to jump to the next match.
Use cgn to change the current match. Type your replacement, then hit <Esc>.
Use n or N to jump to the next or previous match.
Hit . (dot) to repeat the last change on the next match.

For example, to replace foo with bar interactively:

/foo
n
cgnbar<Esc>
n
.
n
n # skip the previous match and don't change it
.

This lets you skip matches you don’t want to change, and quickly apply the same edit to others. You can also use other operators with gn, such as dgn to delete matches, or ygn to yank them.

What’s great about this approach is that the change becomes repeatable with the dot command, making it much faster than running :substitute with confirmation for every match.

Searching and replacing text across buffers with Quickfix list

Sometimes you need to search and replace text across multiple files or buffers. Neovim makes this possible with the Quickfix list and the powerful :cdo and :cfdo commands.

Insert buffers into Quickfix list with `:grep` command

One classic way to batch search and replace is to use the built-in :grep command to populate the Quickfix list. This approach is simple and works out of the box.

To search for foo in all .js files and load the results into the Quickfix list:

:grep foo **/*.js
:copen

This will open the Quickfix window with all matches. You can then use :cdo to run a substitute command on every matching line:

:cdo s/foo/bar/gc

Or use :cfdo to run a command on every buffer in the list:

:cfdo %s/foo/bar/gc

Always review the Quickfix list before running these commands to avoid unwanted changes. You can close the Quickfix window with :cclose.

For more details, see the Neovim Quickfix documentation.

Insert buffers into Quickfix list with picker of `snacks.nvim`

If you prefer a more interactive workflow, modern Neovim plugins like snacks.nvim let you use fuzzy pickers to select files and load them into the Quickfix list. This is my favourite way to batch search and replace, as it gives you full control over which files to operate on.

For example, you can use the grep picker from snacks.nvim to search for a pattern, select the files you want, and send them to the Quickfix list. Once the list is populated, you can use the same :cdo or :cfdo commands as above.

This workflow also works with other pickers like Telescope or fzf-lua. The flexibility to preview and select files before running replacements makes it much safer and more efficient.

How to start Neovim with minimal configuration

Tue, 18 Mar 2025 00:00:00 GMT

In the past, I would install vim alongside nvim in my system as a backup, so I have a terminal editor to fix the configuration of Neovim whenever I broke it and some annoying errors are populated through an autocmd after every keystroke. Since the errors always come from my configuration(init.lua) or the plugins I installed, I wonder if I could start nvim without any configuration, then I can use the vanilla Neovim to fix the issue.

Disable all configuration and plugins with `--clean`

Going through the documentation of Neovim, I notice a --clean option is added in 0.8.0, that would prevent loading configuration files and user’s plugins when starting Neovim. At the same time, built-in plugins for Neovim would be loaded as usual.

nvim --clean

Use an alternative configuration file with `-u`

If you do not like to work with a vanilla Neovim, you can also start it with an alternative configuration file with -u.

nvim -u "$XDG_CONFIG_HOME/nvim/minimal.lua"

I found this much more useful than simply running --clean, as I can run some essential plugins such as nvim-treesitter and nvim-lspconfig in this alternative configuration.

The obvious drawback for this approach is you have to manage another configuration, and you have to make sure it is working.

Adding types to your Neovim configuration

Mon, 10 Feb 2025 00:00:00 GMT

Once again I have started customizing my Neovim’s configuration, so that I can improve my productivity and enjoy every single click on my keyboard at work. After spending hours updating my init.lua and checked out some plugins I have never heard of, I noticed quite a few plugins now include an annotation comment @type for typing their configuration’s arguments.

require("lazy").setup({
    spec = {
        {
          "folke/snacks.nvim",
          priority = 1000,
          lazy = false,

          ---@type snacks.Config
          opts = {

          },
        }
    };
})

As Lua is a weakly-typed language, just like Javascript, adding type support to it is a non-trivial task. Instead of replacing Lua with a typed superset, for example Teal, using annotations is a much more straight-forward approach, as users do not need to rewrite their code and adapt a building process that compiles codes from a typed variant back to Lua. They only need to set up LSP diagnostics support with the editor to enjoy those typings. I believe that is the right approach for improving Lua, given its relative small community.

Rather than getting type suggestions from lua-language-server after adding the annotation, I got the following LSP warning.

Undefined type or alias `snacks.Config`. Lua Diagnostics. (undefined-doc-name)

I tried to require snacks before that type annotation, but lua-language-server still refused to pick up the type. At the end, I found out that I have to update the configuration of lua-language-server, so it can analyze all libraries that are imported in a workspace. As I am using lazy.nvim to manage my plugins, all plugins are installed at $XDG_DATA_HOME/nvim/lazy, so I just need to add that path to the configuration.

require("lazy").setup({
    spec = {
        {
            "neovim/nvim-lspconfig",
            version = "1.6.0",
            event = { "BufReadPre", "BufNewFile" },
            config = function()
                local lspconfig = require("lspconfig")

                lspconfig.lua_ls.setup({
                    capabilities = capabilities,
                    on_init = function(client)
                        client.server_capabilities.semanticTokensProvider = nil

                        if client.workspace_folders then
                            local path = client.workspace_folders[1].name
                            if
                                path ~= vim.fn.stdpath("config")
                                and (
                                    vim.loop.fs_stat(path .. "/.luarc.json")
                                    or vim.loop.fs_stat(path .. "/.luarc.jsonc")
                                )
                            then
                                return
                            end
                        end

                        client.config.settings.Lua = vim.tbl_deep_extend(
                            "force",
                            client.config.settings.Lua,
                            {
                              runtime = {
                                version = "LuaJIT",
                              },
                              workspace = {
                                checkThirdParty = false,
                                library = {
                                  "$VIMRUNTIME",
                                  "$XDG_DATA_HOME/nvim/lazy", 
                                  "${3rd}/luv/library",
                              },
                            },
                        })
                    end,
                })
            end,
        },
    },
})

After updating my configuration, the LSP is able to pick up types and give suggestions, even without an explicit type annotation.

Avoid global configuration with `.luarc.json`

Configuration for lua-language-server in init.lua is global, and it will load and analyze types from $XDG_DATA_HOME/nvim/lazy, even if your project does not require any module from it, slowing down the LSP server start up time. A better option is to use .luarc.json to customize lua-language-server. By creating this configuration file at the root level of your project, only libraries specific to that project will be analyzed.

{
  "$schema": "https://raw.githubusercontent.com/LuaLS/vscode-lua/master/setting/schema.json",
  "workspace.library": ["$VIMRUNTIME", "$XDG_DATA_HOME/nvim/lazy", "${3rd}/luv/library"]
}

Conditionally add values into list or map in Nix

Wed, 05 Feb 2025 00:00:00 GMT

I often find myself conditionally adding values into a list or map when writing configurations and derivations in Nix. In this post I want to show you how to conditionally add values into list or map in Nix.

Similar to many other functional languages, all values are immutable, and no statement exists in Nix. Instead of relying on method such as push() that would mutate an object, you have to use composition to add values into a list or map.

Conditionally add values into a list in Nix

To conditionally add values into a list in Nix, we can use the list concatenation operator ++ together with a if expression. Remember if in Nix is an expression instead of statement, therefore it can return value. The following example shows you how to add c and d into the list of foo, if acceptAdditional is true.

let
   acceptAdditional = true;
in {
  foo = [ "a" "b" ] ++
    (if config.bar.enable then [ "c" "d" ] else [ ]);
}

If we evaluate the above file with nix repl, we can see that the value of foo evaluated to the combination of foo and bar.

❯ nix repl --file example.nix
Welcome to Nix 2.18.1. Type :? for help.

Loading installable ''...
Added 1 variables.
nix-repl> foo
[ "a" "b" "c" "d" ]

We can simplify our code with the helper function lib.lists.optionals for the same effect, where an empty list will be returned, if the condition is not true.

let
   acceptAdditional = true;
in {
  foo = [ "a" "b" ] ++
    (if acceptAdditional then [ "c" "d" ] else [ ]);
    (lib.lists.optionals acceptAdditional [ "c" "d" ]);
}

Conditionally add values into a map in Nix

To conditionally add values into a map in Nix, we can use the Update operator //. The attributes of the map in the right-hand side of the operator, will be merged into the map in the left-hand side.

let
    acceptAdditional = true;
in {
  foo = {
    a = "d";
    b = "e";
  } // (if acceptAdditional then {
    one = "1";
    two = "2";
  } else
    { });
}

By evaluating it, we can see that attribute one and two is now merged into foo.

❯ nix repl --file example.nix
Welcome to Nix 2.18.1. Type :? for help.

Loading installable ''...
Added 1 variables.
nix-repl> foo
{ a = "d"; b = "e"; one = "1"; two = "2"; }

If any attribute on the right-hand side’s map is identical with the left-hand side’s map, it will overwrite its value. This is really useful for conditionally setting value of attributes in a map.

let
    acceptAdditional = true;
in {
  foo = {
    a = "d";
    b = "e";
  } // (if acceptAdditional then {
    one = "1";
    two = "2";
    a = "new value";
  } else
    { });
}

As you can see, the value of a has been overwritten into the value of the same attribute of the right-hand side map.

❯ nix repl --file example.nix
Welcome to Nix 2.18.1. Type :? for help.

Loading installable ''...
Added 1 variables.
nix-repl> foo
{ a = "new value"; b = "e"; one = "1"; two = "2"; }

To simplify our code and avoid writing if expression, we can use the helper function lib.attrsets.optionalAttrs.

let
    acceptAdditional = true;
in {
  foo = {
    a = "d";
    b = "e";
  } // (lib.attrsets.optionalAttrs acceptAdditional {
    one = "1";
    two = "2";
    a = "new value";
  });
}

The limitation for \\ operator is that it cannot merge recursively. If you want a recursive merge, you can use the helper function lib.attrsets.recursiveUpdate.

Practical use cases for adding values conditionally

plymouth is an application that allows you to display a custom splash screen when you boot a Linux machine. To make plymouth useful and avoid disrupting the splash screen with logs, we have to set quiet and splash at boot.kernelParams.

Instead of setting it directly at boot.kernelParams, and comment it out when we disable plymouth, we can use the aforementioned technique, and add those parameters to boot.kernelParams when config.boot.plymouth.enable is true.

{ config, pkgs, lib, inputs, ... }:

{
  boot.plymouth.enable = true;
  boot.kernelParams = []
    ++ (lib.lists.optionals config.boot.plymouth.enable [ "quiet" "splash" ]);
}

Optimize SvelteKit performance with brotli compression

Sun, 02 Feb 2025 00:00:00 GMT

In my post of building Docker image for static SvelteKit application with nginx, I have covered almost everything, except serving brotli compressed assets. brotli generally compress files better than traditional gzip.

Today I want to share how to optimize SvelteKit’s performance with brotli compression, and serve those pre-compressed assets with nginx.

Enable pre-compression in SvelteKit

As we are building a static site, we can compress all our assets with brotli during the build time. To enable pre-compression, you need to set precompress: true in your adapter.

import adapter from '@sveltejs/adapter-static';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  kit: {
    adapter: adapter({
      precompress: true
    }),
  },
};

export default config;

Enable `brotli` support in `nginx`

By default, nginx does not support brotli compression, and it won’t serve pre-compressed brotli assets as well. To enable that, you have to install ngx_brotli module.

Luckily there is a Docker image with ngx_brotli pre-installed. We are going to modify our Docker image that build SvelteKit with static adapter and serve with nginx, and use that image.

# omitted for brevity
FROM base AS prerelease
COPY --from=install /temp/dev/node_modules node_modules
COPY . .

ENV NODE_ENV=production
RUN npm run build

# the nginx Docker image with brotli modules installed
FROM KiweeEu/nginx-brotli AS release
COPY --from=prerelease --chown=nginx:nginx /usr/src/app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

And finally we just need to allow nginx to serve brotli compressed content, by modifying nginx.conf.

# load the ngx_brotli module
load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_brotli_static_module.so;

server {
    # some usual nginx config
    listen 8080;
    listen [::]:8080;
    root /usr/share/nginx/html;
    server_name _;
    try_files $uri $uri.html $uri/ =404;

    brotli_static on; # allow serving brotli compressed files 
}

Using glob on CLI that does not support variadic arguments

Fri, 31 Jan 2025 00:00:00 GMT

Recently I need to use sops to encrypt and decrypt a set of secret files for deploying Kubernetes workload using the GitOps approach. All of these files has an extension of sops, for example secret.sops.yaml. For a simple task like this, I was expecting everything can be done in a single line using glob.

sops --encrypt **/*.sops.* # encrypt all files with a sub extension of sops

Unfortunately, sops cannot handle variadic arguments. Only the first argument is handled, and the rest are simply ignored. As a glob will be expanded by the shell into a space separated string, and array in shell can be created based on a space separated string as well, I wonder if I can turn a glob into an array, and then iterate all its items in a loop.

I started off by verifying that I can build an array from glob, using a really simple example.

touch foo.md bar.md
arr=(*.md)
echo "${#arr[@]}" # getting the length of the array, and it prints 2

And then, I tried to run a for loop on this array, and it worked.

for file in $list[@]; do echo "$file"; done # foo.md bar.md

I thought I have reached an optimal solution already, until I found out that you can use for loop without an array! We can use glob directly on a for loop, and everything is nice and clean.

for file in *.md; do echo "$file"; done # foo.md bar.md

Allowing glob to fail gracefully

Everything seems to be working so far. I have then implemented my script in the actual project, and found out that my script will break when the glob pattern cannot match anything. I thought it would be expanded to an empty string, yet it exited right away.

After searching for a while, I found that you could set shopt -s nullglob in bash and zsh and prevent an unmatched glob from exiting.

End-to-end type safety with Svelte 5 and SvelteKit 2

Mon, 27 Jan 2025 00:00:00 GMT

Back in Svelte 4 and SvelteKit 1, you can use PageData and LayoutData to annotate the type of data passed from a load function in +page.svelte and +layout.svelte respectively to achieve end-to-end type safety. These two types are generated by SvelteKit based on the value returned by the load function.

For example, assuming we fetch data from an API in a load function as the following.

import type { PageServerLoad } from './$types';

export const load: PageServerLoad = async ({ params, route, parent }) => {
    const { cat, dog } = await getAnimals() // cat has type Cat, dog has type Dog

    return {
        cat: cat,
        dog: dog
    };
};

You can then import PageData to annotate the data props in your component.

<script context="module" lang="ts">
	import type { PageData } from './$types';
</script>

<script lang="ts">
	export let data: PageData; // the type here will be { cat: Cat, dog: Dog } 
</script>

Type safety with `$props()` rune in Svelte 5

In Svelte 5, with the introduction of $props() rune, every Svelte component only accepts a single object as its props. Since the type of $props() returns any by default, and does not accept type parameters, you have to add a type annotation to the variable to achieve type safety.

<script context="module" lang="ts">
    import type { PageData } from './$types';
</script>

<script lang="ts">
    export let data: PageData;
    let { data }: { data: PageData } = $props();
</script>

Simplify type annotation with `PageProps`

With the release of SvelteKit 2.16.0, new types PageProps and LayoutProps are introduced to simplify the type annotation for props in +page.svelte and +layout.svelte respectively.

<script context="module" lang="ts">
    import type { PageData } from './$types';
    import type { PageProps } from './$types';
</script>

<script lang="ts">
    let { data }: { data: PageData } = $props();
    let { data }: PageProps = $props();
</script>

Under the hood, PageProps and LayoutProps reuse PageData and LayoutData in their definition, nothing too special there.

export type PageProps = { data: PageData; form: ActionData };

Recreating development environment with Nix Flake

Thu, 23 Jan 2025 00:00:00 GMT

In this post, I will show you how to define and recreate development environment of your projects anywhere using Nix Flake.

Create your development environment with `nix develop`

You can define the development environment of your project in Nix Flake under devShells with pkgs.mkShell, and a shell based on that environment can be created with nix develop .#<environment-name>. For the packages you want to include in this environment, you can define them in nativeBuildInputs, and they will be loaded through the PATH in the shell automatically.

The following example is a flake that install kustomize in devShells.default. flake-parts is used to simplify configuration, so you can define devShells.<name> instead of devShells.<system>.<name> for each system.

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };

  outputs = inputs@{ self, nixpkgs, flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" ];

      perSystem = { config, pkgs, system, ... }:
        let
          pkgs = import nixpkgs {
            inherit system;
            config.allowUnfree = true;
            overlays = [ ];
          };
        in {
          devShells.default = pkgs.mkShell {
            nativeBuildInputs = with pkgs; [
              kustomize
            ];
          };
        };
    };
}

If you want to define environment variables in this environment or trigger some actions when entering it, for example start a database process in background, you can define those commands in shellHook.

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };

  outputs = inputs@{ self, nixpkgs, flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" ];

      perSystem = { config, pkgs, system, ... }:
        let
          pkgs = import nixpkgs {
            inherit system;
            config.allowUnfree = true;
            overlays = [ ];
          };
        in {
          devShells.default = pkgs.mkShell {
            nativeBuildInputs = with pkgs; [
              kustomize
            ];

            shellHook = ''
              set -a;
              export FOO="bar";
              set +a;
            '';
          };
        };
    };
}

You might have also seen snippets of pkgs.mkShell that defines what packages to use through buildInputs or packages. buildInputs is interchangeable with nativeBuildInputs, unless you are cross compiling, as packages defined in it are for the foreign platform that your compilation is targeting, and packages defined in nativeBuildInputs are for the native platform where the compilation will happen. packages is just an alias argument for nativeBuildInputs, as it will be merged with the nativeBuildInputs at the end.

Under the hood, pkgs.mkShell creates a derivation that only has a build phrase, which does not do anything on its own, and it is meant to be used only with the old nix-shell command, that is wrapped by nix develop.

Automate environment creation with `direnv`

Instead of running nix develop manually every time, what if there is a tool that can automate it for you, whenever you navigate into that project? direnv is a tool that does exactly that. With its Nix Flake integration, nix-direnv, it would call nix develop for you when you cd into a project, and the PATH and environment variables in your current shell will be updated temporarily, until you navigate away from that project. This is a small but important improvement for me, as I don’t have to use a bare-boned bash shell and I can continue to enjoy the custom prompt and key bindings in my current shell profile.

There are a few ways to install nix-direnv in your system. Assuming you are using Home Manager and you are using zsh as your shell, you would create a module as the following. Similar configuration exists for bash and fish as well.

{ inputs, lib, config, pkgs, system, isDarwin, ... }: {
  programs.zsh = {
    enable = true;
  };
  programs.direnv.enable = true;
  programs.direnv.enableZshIntegration = true;
  programs.direnv.nix-direnv.enable = true;
}

Then in the project you want to enable nix-direnv support, you need to create a file called .envrc with use flake.

use flake

Instead of the default devShells, you can define and run a specific devShells. In the following example, we instruct direnv to run devShells.<system>.ci automatically for us.

use flake .#ci

Finally, you will need to run direnv allow once at the project level to allow direnv to do its magic. The next time you navigate into this project, your development environment will be recreated automatically by direnv.

Reproduce your development environment anywhere with Docker

Since our development environment is defined in Nix Flake, we can reproduce our environment on any machine with Nix Flake support, including a Docker image. The Nix community provides a Docker image for running Nix Flake, and we can recreate our project environment and execute commands in it.

I found it very useful to bring my environment from Nix Flake into a CI/CD pipeline through Docker. Not only the version of packages I get are identical with what I have locally, but also I don’t need to define the installation process for each of them again. Everything is controlled in the Nix Flake.

Since we cannot interact with a shell started by nix develop in a pipeline, we have to pass the command and arguments that we want to execute to nix develop as nix develop -c <command> [...args]. The following example is a task for Tekton pipeline that executes kustomize build --enable-helm ./final in an environment created by nix develop.

---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: generate-manifest
spec:
  steps:
  - image: docker.io/nixpkgs/nix-flakes:nixos-24.11
    name: generate-manifest
    script: |
      nix develop -c kustomize build --enable-helm ./final;

Analyze and optimize your Vite bundle

Sat, 18 Jan 2025 00:00:00 GMT

It is important to analyze your bundle generated by Vite from time to time, as often there will be some low-hanging opportunities to reduce its size. In this post I will show you how to analyze and optimize your Vite bundle by removing duplicate code and packages.

Analyze your Vite bundle

I have been using vite-bundle-visualizer in the past. It does what it says in the box and visualize my bundle, but I still have to spend time on examining charts and figure out what issues does my bundle contain.

I think rollup-plugin-bundle-stats addressed my pain point here, and it is a superior option to analyze your Vite bundle. It gives me a clear pointer on duplicate code and packages, so I can address them easily. As Vite uses Rollup for bundling, we can use this plugin with Vite as well. You can install this package with whatever package manager you want.

npm i rollup-plugin-bundle-stats -D

After installing this package, you need to add this plugin to your Vite configuration. Even though all options for this plugin is optional, it cannot find the baseline.json in my SvelteKit project out of the box, and I have to set the path manually at baselineFilePath. I have also set silent: true to reduce the log output from this plugin.

import { defineConfig } from 'vite';
import { bundleStats } from 'rollup-plugin-bundle-stats';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';

const currentDir = dirname(fileURLToPath(import.meta.url))

export default defineConfig({
  plugins: [

    bundleStats({
      baselineFilepath: join(currentDir, "node_modules", ".cache", "bundle-stats", "baseline.json"),
      silent: true,
    }),
  ],
});

Then you can run vite build to build your project and generate the package analysis. You should be able to find a bundle-stats.html in the output directory of vite build.

Remove duplicate packages from your bundle

To walk you through the process of removing duplicate packages from your bundle, I have created a minimal SvelteKit project with @grafana/faro-rollup-plugin, @grafana/faro-web-sdk and @grafana/faro-web-tracing installed. These packages have some shared and duplicated dependencies, and it is perfect for demonstration.

Running vite build for this project for the first time, and inspect bundle-stats.html, we can see that there are 7 duplicate packages in this project. Click on the “duplicate packages” tab, we can see that the duplicate packages are @opentelemetry/core, @opentelemetry/semantic-conventions and @opentelemetry/sdk-trace-web.

To unify the version of our packages and remove duplicate, we can define the desired version in package.json in overrides (or resolutions if you are using yarn). After updated package.json, we then need to run npm dedupe to update package-lock.json and remove duplicate packages, and we can run vite build again to verify the result. You might have to repeat this process multiple times, until you have removed all duplicate, as the new resolved packages might have duplicate dependencies with your existing packages.

A word of warning, overriding packages’ version with overrides might break your project. It is better to write some tests beforehand to verify features of your project is still intact.

{
  // ...
  "devDependencies": {
    "@grafana/faro-rollup-plugin": "^0.1.1",
    "@grafana/faro-web-sdk": "^1.12.2",
    "@grafana/faro-web-tracing": "^1.12.2"
  },

  "overrides": {
    "@opentelemetry/resources": "1.27.0",
    "@opentelemetry/core": "1.27.0",
    "@opentelemetry/semantic-conventions": "1.27.0",
    "@opentelemetry/sdk-trace-web": "1.27.0"
  }
}

At the end, we could see that all duplicate packages are gone, and the bundle size decreased slightly as well. With a bigger project, you should see greater changes.

Reduce input latency for gaming with Gnome in NixOS

Fri, 17 Jan 2025 00:00:00 GMT

Recently I have started gaming in NixOS with Gnome, and everything is running smooth and slick, except a noticeable input latency, which is unbearable as I was playing shooter game.

After searching for a while, I realized that input latency can be reduced by enabling variable refresh rate (VRR). Gnome has added in experimental support for VRR starting from version 46, and it is disabled by default.

To enable this experimental feature, you have to use gsettings or dconf to add variable-refresh-rate as an item of the value of /org/gnome/mutter/experimental-features.

# if you prefer gsettings
gsettings set org.gnome.mutter experimental-features "['variable-refresh-rate']"
# if you prefer dconf
dconf write /org/gnome/mutter/experimental-features "['variable-refresh-rate']"

To do this the Nix way, if you are using Home Manager and you want to enable this feature on a user basis, you set this value through dconf.settings.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/mutter" = {
      experimental-features = [ "variable-refresh-rate" ];
    };
  };
}

Creating Content Security Policy in SvelteKit

Thu, 16 Jan 2025 00:00:00 GMT

What is Content Security Policy (CSP)?

In a nutshell, Content Security Policy (CSP) is a list of trusted origins which only their resources can be loaded on your website once defined. This is a security feature for preventing attacks such as cross-site scripting (XSS) and data injection.

CSP can be defined through HTTP response headers Content-Security-Policy and Content-Security-Policy-Report-Only or meta tags with <meta http-equiv>.

Setting up Content Security Policy in SvelteKit

SvelteKit provides CSP support out of the box. We just need to define the policy we want through kit.csp, and SvelteKit will generate the corresponding hash or nonce for its inline style and scripts, and policies as <meta http-equiv> tag in HTML.

Initially I defined my CSP based on the basic non-strict policy from OWASP. default-src directive would be used as a fallback for most unspecified directives, except a few exceptions such as frame-ancestors and form-action directive. The values of a directive are the allowed origins, and self represents the same origin as your website.

By defining our policies in kit.csp.reportOnly instead of kit.csp.directives, browsers would not block non-compliant resources but only log violations or make a POST request to the endpoint defined in report-uri. With that information, you can discover resources your website depending on, and gradually tighten your security policies without breaking anything, by moving your security policies from kit.csp.reportOnly to kit.csp.directives.

const config = {
  kit: {
    csp: {
      // NOTE without setting a truthy value for kit.csp.directives, SvelteKit would not do anything for CSP
      directives: {},
      reportOnly: {
        "default-src": ["self"],
        'frame-ancestors': ['self'],
        'form-action': ['self'],
        'report-uri': ['/'],
      },
    },
  },
};

I have created a minimal SvelteKit project with these content security policies defined, so you can play around yourself.

Dealing with CSP violations in SvelteKit

The above configuration should work for most of the SvelteKit projects. From time to time you might encounter packages that violates your CSP. For example, on this website I use Grafana Faro for tracking performance my website. The SDK for this service, @grafana/faro-web-sdk used a package contains eval() in its code, I then notice the following log in my console.

Content-Security-Policy: (Report-Only policy) The page’s settings would block a JavaScript eval (script-src) from being executed because it violates the following directive: “script-src 'self' 'nonce-ewcvSmSO/2WM8RPgzlYn3A==' (Missing 'unsafe-eval')

An option to address this is to add unsafe-eval to script-src directive as suggested, but this is a global setting for my website, not only for this package. We won’t get much benefit from CSP if we had used unsafe-eval or unsafe-inline.

The other option is we move the policies to kit.csp.reportOnly for reporting violations only, and we monitor those violations reports to verify our website is secure or not. You could host a server that receives and converts the POST payload sent by Reporting API from browsers to metrics understandable by tools such as Prometheus or OpenTelemetry, and then set up alerting rules to monitor content security policy’s violations. This approach requires more work in the back end, but it saves you from refactoring code or switch packages in your SvelteKit project.

From my research, I found go-csp-collector for collecting the reports, and it seems to be a good starting point for building your monitoring system.

How to get 100 performance score in Lighthouse with SvelteKit

Mon, 13 Jan 2025 00:00:00 GMT

Lighthouse is a tool for auditing performance, accessibility and SEO of a web page. As I want to give my readers the best browsing experience, I am concerned with the Lighthouse score of my website.

In this post, I will show you a few techniques on optimizing your SvelteKit project for getting 100 performance score on Lighthouse.

Minifying HTML response

Out of the box, only CSS and Javascript is minified by SvelteKit. HTML is not handled.

To minify HTML responses, no matter which adapter you are using, you can transform the HTML response in hooks.server.js or hooks.server.ts with an HTML minification library. The following is a snippet of using html-minifier to minify the HTML response.

import { minify } from 'html-minifier';
import { type Handle } from '@sveltejs/kit';

const minifyOpts = {
  collapseBooleanAttributes: true,
  collapseWhitespace: true,
  conservativeCollapse: true,
  decodeEntities: true,
  html5: true,
  ignoreCustomComments: [/^#/],
  minifyCSS: true,
  minifyJS: false,
  removeAttributeQuotes: true,
  removeComments: false,
  removeOptionalTags: true,
  removeRedundantAttributes: true,
  removeScriptTypeAttributes: true,
  removeStyleLinkTypeAttributes: true,
  sortAttributes: true,
  sortClassName: true,
};

export const handle: Handle = async ({ event, resolve }) => {
  return resolve(event, {
    transformPageChunk: ({ html, done }) => {
      return minify(html, minifyOpts);
    },
  });
};

Enable pre-compression

After minifying your assets, you should also compress them with gzip or brotli, reducing the size of files to be transmitted. Some adapters, such as @sveltejs/adapter-node and @sveltejs/adapter-static support pre-compression, and you can pre-compress your assets in the build time. Otherwise, you will have to compress your assets on the fly.

import adapter from '@sveltejs/adapter-static';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  kit: {
    adapter: adapter({
      precompress: true,
    }),
  },
};

export default config;

After compressing your assets, you might need to configure your load balancer for serving them. My guide on serving compressed files with nginx might be useful for you.

Cache immutable assets with `Cache-Control` HTTP response header

If you have ever examined the build output of SvelteKit, you may notice that /_app/immutable contains all your CSS and Javascript of a build (except service worker script, sw.js), and every file in it has a hash in their file name, which is generated based on the content of the file. These hashes are used for cache-busting, and files with such hash should be cached for as long as possible.

Cache-Control HTTP response header indicates how long a resource should be cached. You should at least cache files in /_app/immutable for a year.

Cache-Control: max-age=31536000

Do not add such strong caching rule to a file without hash, unless you know what you are doing. Setting Cache-Control: no-cache to those files is a good default, which the browser will still cache the file, but always revalidate, making sure the latest version is served.

Depending on what you prefer, you can set this header at your load balancer level or file server level. The configuration depends on what you use. If you are using nginx, you can add this header inside a location block.

server {
    # omitted for brevity

    location /_app/immutable {
        add_header Cache-Control "max-age=31536000";
    }
}

Optimize images

This is the most important optimization to have, as an unoptimized image can drag down the performance of your website significantly. In principle, optimizing images on a webpage means serving the right format and the right size. You should use format that can be compressed smaller such as webp or avif, and you should serve a smaller image, if your user is browsing your website with his phone. In a nutshell, you have to generate images in multiple formats and dimension to achieve this.

Optimize images at build time

In the past I have to set up tools such as sharp and create an image markup component to load the generated images and the original image as fallback, using <picture> and <source>. Fortunately, the new experimental plugin @sveltejs/enhanced-img do all that for us.

<script>
import FooImage from './foo.jpg?enhanced'; // import the source image but generated the enhanced version as well
</script>

<!-- this component is using <picture> and <source> as well, just nicely packaged -->
<enhanced:img src={FooImage} alt="some alt text" />

Optimize images at run time

If your images are stored in a S3 bucket or somewhere remote, and you load them dynamically, you can only optimize them in runtime. You can use an image transformation proxy, such as imgproxy to generate an optimized image on the fly.

As an alternative, you can also run scripts that optimize images continuously with cron jobs.

Enable pre-rendering if possible

By pre-rendering your pages, you create an HTML file with all the variables in your Svelte markup substituted in the build time. Users of your application will then get a hydrated (basically it means completed) HTML file immediately and reduce First Contentful Print. You can set const prerender = true in your +layout.ts or +page.ts to enable pre-rendering. Setting it at +layout.ts will enable pre-rendering for all pages using that layout.

export const prerender = true;

However, pre-rendering might not be useful to your application, if your users cannot benefit from stale data. Say your application shows real time stocks’ price, pre-rendering and shipping the prices at the build time to users initially is meaningless.

Disable client side rendering (CSR)

If your page can be pre-rendered or server side rendered, and you do not need interactivity on your page, you can simply disable client side rendering and not ship Javascript to your users. This would obviously reduce the payload you send to your users, but it might only be feasible for static pages, such as a blog post.

To disable CSR, you can set const csr = false in a +layout.ts or +page.ts, and it will disable CSR for a set of pages using that layout or an individual page.

export const csr = false;
// enable at least ssr or pre-rendering, otherwise a blank page will be shipped
export const ssr = true;
export const prerender = true;

Optimize font rendering

If no font specified through font-family has been loaded, browsers would wait for it and would not render text for a period of time. To make browsers fallback to system font, until at least one specified font is available, you should set font-display: swap in your @font-face rule in CSS.

@font-face {
  font-family: Roboto;
  font-display: swap;
}

Also you should only use font with woff2 format, as it can achieve better compression comparing with other formats, and it is supported by almost all browsers in 2025.

If you don’t want to deal with all these manually, I would recommend using font packages from Fontsource. Out of the box they set font-display: swap and use only woff2 font files.

Synchronizing inputs across Nix flakes

Sat, 11 Jan 2025 00:00:00 GMT

I have a few projects using nix-direnv to load development environments automatically, and each of them is using a slightly different version of nixpkgs, depending on when I created them. As my laptop with 256 GB storage run out of space again, and I do not depend on any unstable software with my projects, I want to synchronize their inputs to avoid duplications and reduce storage usage.

Synchronizing inputs with `nix registry`

As I understand it, nix registry is just an alias to an actual input. By using an alias, you can then force multiple flakes to use the exact same version of an input.

You can define your own nix registry specific to your machine with nix.registry.

To create a registry that is called foo, and points to nixos-23.05 branch in NixOS/nixpkgs repository on GitHub, you can update your NixOS configuration as the following.

{ config, pkgs, lib, ... }:

{
    nix.registry = {
        foo.to = {
            owner = "NixOS";
            ref = "nixos-23.05";
            repo = "nixpkgs";
            type = "github";
        };
    };
}

And then you can use this foo as input in Nix flake. You do not need to define this variable in inputs, as missing inputs will be resolved by the registry automatically.

{
  outputs = { self, foo }:
    {

    };
}

Using nix.registry can only create nix registry on your local machine. If you want to create a registry definition that can be shared among multiple machines, you can create a JSON file according to the format accepted by nix registry and host it somewhere, and point to that JSON file in the nixConfig.flake-registry in your flake.nix. A good start will be forking the global flake registry.

{
  outputs = { self, foo }:
    {

    };

  nixConfig = {
    flake-registry = "https://example.com/myrepository.json";
  };
}

In my opinion you should always have nixConfig.flake-registry defined if you are using nix registry. Without defining nixConfig.flake-registry, nix registry will become a global mutable state. If you cannot control the nix registry definition in your flake consumers’ machine, you cannot guarantee your flake would build there. What if foo points to nixos-23.05 on your machine, but foo points to nixos-14.05 on their machine? What if you have two projects using the same registry name in their flake, but that name should resolve to two entirely different inputs?

Synchronizing inputs with proxy flake

Another option to synchronize inputs is to create a proxy flake. It will be used as an input by your downstream flakes and its inputs will be followed, exposing them to downstream flakes. This approach obviously works with multiple machines.

For example, I have a flake that has the following inputs as proxy.

{
  description = "proxy flake for controlling input for all my flakes";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
    flake-parts = {
      url = "github:hercules-ci/flake-parts";
      inputs.nixpkgs-lib.follows = "nixpkgs";
    };

    nixos-hardware.url = "github:NixOS/nixos-hardware/master";

    nur.url = "github:nix-community/NUR/master";
    nur.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs }:
    {

    };
}

And then I can use its inputs indirectly in my projects’ flake.

{
  inputs = {
    proxy-flake.url = "github:winston0410/proxy-flake?main";
    nixpkgs.follows = "proxy-flake/nixpkgs";
    flake-parts.follows = "proxy-flake/flake-parts";
  };

  outputs = { self, nixpkgs, flake-parts }:
    {

    };
}

Personally I prefer using proxy flake over nix registry, as it completely remove the possible pitfall of a global mutable state.

Upgrading inputs

To update inputs, after changing their target commit or branch in your registry definition or proxy flake, you still have to run nix flake update manually to update downstream flakes. Even though it seems to be a hassle, inputs of your projects would not update automatically without you knowing. But of course you can write a script to automate it.

No matter which approach you have chosen, you can still find sha256 hash in flake.lock to confirm which commit or branch does it resolve to.

Convert git repository into a tarball for sharing

Fri, 10 Jan 2025 00:00:00 GMT

A new hire joined our company, but the engineer who manages role and permission for the company is still on holiday. Obviously he didn’t get access to our remote git repository, so I had to convert our repository into a tarball and send it to him.

In this post, I will share how can you convert git repository into a tarball that respects .gitignore for easy sharing.

Creating tarball from remote repository

After searching for a while, I came across this Stackoverflow answer and learnt about git archive. This command creates an archive of files from a commit, therefore .gitignore is respected and only commited files will be included in the archive. Possible output format supported are tar, zip, tar.gz and tgz.

Since git archive is commit-based, you can reference your remote git repository to create an archive.

git archive --format=tar -o ~/myarchive.tar --remote=https://mygitrepository/myproject.git master # it accepts branch, commit or tag

Be mindful that this command would only work if the target git server supports it. Also if you want to use http instead of git protocol for your remote git repository, you have to use git 2.44.0 or later.

Creating tarball from local repository

If archiving directly from remote git repository is not an option for you, you can still clone the repository first, and then create an archive locally.

git archive --format=tar -o ~/myarchive.tar master

Creating sitemap for your SvelteKit project with minimal effort

Thu, 09 Jan 2025 00:00:00 GMT

Sitemap is crucial for the SEO success of a website. Even though SvelteKit is designed with SEO in mind, the decision was to keep sitemap generation in the userland. I have been using SvelteKit since version pre 1.0, and I have created sitemaps for SvelteKit projects using different adapters. In this post I want to share my experience on building sitemap in SvelteKit and introduce super-sitemap to you, which provides the ideal interface and experience for building sitemap in my opinion.

Common ways of creating sitemap for SvelteKit projects

Building sitemap usually involve two steps, metadata retrieval and transformation. The process of metadata retrieval is largely shaped by how data is stored, and it is specific to a project. Projects that store metadata as local files usually use import.meta.glob function for bulk import; projects that store metadata remotely, such as in a CMS, usually collect its data with HTTP requests.

After metadata are collected, it should then be transformed into structures that you would expect to see in a sitemap. It could be transformed directly as <url> nodes and embedded in a sitemap as string, or as objects that will be consumed by sitemap generating libraries. No matter how you do it, this step is relatively generic.

import type { RequestHandler } from '@sveltejs/kit';
import { loadPosts } from '$lib/content';
export const GET: RequestHandler = async ({ url }) => {
  // this function could load from local files with import.meta.glob() or fetching data from remote, depending on how you store your data
  const posts = loadPosts();

  return new Response(
    `
<?xml version="1.0" encoding="UTF-8" ?>
<urlset
  xmlns="https://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xhtml="https://www.w3.org/1999/xhtml"
  xmlns:mobile="https://www.google.com/schemas/sitemap-mobile/1.0"
  xmlns:news="https://www.google.com/schemas/sitemap-news/0.9"
  xmlns:image="https://www.google.com/schemas/sitemap-image/1.1"
  xmlns:video="https://www.google.com/schemas/sitemap-video/1.1"
>
${posts
  .map(({ id, updatedAt }) => {
    return `<url>
              <loc>${url.origin}/blog/${id}</loc>
              <lastmod>${updatedAt}</lastmod>
              </url>`;
  })
  .join('')}
</urlset>`.trim(),
    {
      headers: {
        'Content-Type': 'application/xml',
      },
    },
  );
};

This approach is imperative, and the pitfall for it is you have to duplicate your routing configuration in code and maintain them, as metadata do not have information on what routes your SvelteKit project has. Say the id of your posts are all slugified strings, such as my-post-title, and in your SvelteKit project you have a route /blog/[slug]. You have to transform your data and prepend /blog when you build your sitemap. A few months later, your project manager decided to add i18n support in your website, and you changed the route to /[lang]/blog/[slug]. I bet you a pound to penny someone will forget to update the sitemap.

Creating sitemap for SvelteKit projects with `super-sitemap`

The cleanest and the easiest way to create a sitemap in SvelteKit right now is probably with the super-sitemap. Instead of using import.meta.glob function to import all the data files, super-sitemap only imports route definition files such as +page.svelte, +page.md and +page.svx and tracks all routes you have in your project. You have to exclude or provide metadata for all route parameters for every route, otherwise it will throw error.

Building sitemap with super-sitemap is more declarative, as it expects you to provide values for route parameters found in your routes only, and path building is managed by it. You do not need to duplicate routing configurations in code.

import * as sitemap from 'super-sitemap';
import type { RequestHandler } from '@sveltejs/kit';
import { loadPages, loadPosts } from '$lib/content';
export const GET: RequestHandler = async ({ url }) => {
  // you can pull metadata from wherever you like. It could be from `import.meta.glob` or from a CMS.
  const posts = loadPosts({
    pinned: false,
    description: true,
  });

  return await sitemap.response({
    origin: url.origin,
    excludeRoutePatterns: [
      // you can exclude routes from the generated sitemap.
      '^/author.*',
      '^/tag.*',
      '^/archive.*',
    ],
    paramValues: {
      // and you provide your metadata to super-sitemap, using route with route parameter as key, so it can substitute those routes for you in the sitemap output.
      '/blog/[slug]': posts.map((x) => {
        return {
          values: [x.id],
          lastmod: x.updatedAt,
        };
      }),
    },
  });
};

With super-sitemap covering route tracking and sitemap generation, you can focus on retrieving metadata for your routes. This separation of concerns makes the project easier to refactor and maintain down the road.

Simplify your C# code with `using` statement

Wed, 08 Jan 2025 00:00:00 GMT

As someone who started coding with Javascript and now mainly working with C#, my transition was smooth as these two languages are relatively similar syntax wise. As I read more and more C# code written by my colleagues, I realised how concise my code can be using the using statement.

In this post I will share what is using statement, when can you use it and its limitation.

What is a `using` statement and when can you use it

The using statement is a feature introduced since C# 8.0, and it can only be used with objects that implemented System.IDisposable or System.IAsyncDisposable. It is a syntactical sugar that will be transformed to a try...finally statement during compile time, and StreamReader.Dispose method will be automatically called in the finally block.

Considering an example of creating a connection with a SQL database and executing SELECT GETDATE(), using the try...finally statement.

SqlConnection conn = null;
try
{
    conn = new SqlConnection("Server=whatever;Database=whatever;User Id=whatever;Password=whatever;");
    await conn.OpenAsync();
    SqlCommand command = new SqlCommand("SELECT GETDATE()", conn);
    object result = command.ExecuteScalar();
}
finally
{
    if (conn != null)
    {
        await conn?.CloseAsync(); // close connection so we don't occupy all of them in the pool
    }
}

If we rewrite that with the using statement, the finally block is now hidden from us. Also we don’t have to declare a variable outside the scope of try...finally block, and reassign the variable. Our code is now clean and concise.

using(var conn = new SqlConnection("Server=whatever;Database=whatever;User Id=whatever;Password=whatever;")){
    await conn.OpenAsync();
    SqlCommand command = new SqlCommand("SELECT GETDATE()", conn);
    object result = command.ExecuteScalar();
}

Handling exception with the `using` statement

You might ask where is the catch? Since using statement is only a syntactical sugar for try...finally, it does not catch exception for you. To catch exception, you will have to put using statement inside try...catch.

try
{
    using(SqlConnection conn = new SqlConnection(connString))
    {
      // interact with the connection, omitted for brevity
    }
}
catch (Exception exc)
{
    // handle exception
}

I have also seen people putting try...catch inside using too, but I would not recommend this(unless you want to lay mines for your team). Since the try...catch is now inside the scope of using, exceptions of SqlConnection conn = new SqlConnection(connString) are not caught. You have to make sure that line would not throw exception if you write it this way.

using(SqlConnection conn = new SqlConnection(connString))
{
    try
    {
        // interact with the connection, omitted for brevity
    }
    catch (Exception exc)
    {
        // handle exception
    }
}

Customizing Gnome with Nix and Home Manager

Mon, 06 Jan 2025 00:00:00 GMT

In this post, I will share my experience and some snippets on customizing Gnome using Nix and Home Manager.

Remove utilities installed with Gnome

If you want to start with a minimal system, you can remove all the utilities installed alongside Gnome by default.

{ config, pkgs, lib, inputs, ... }:

{
  # omitted for brevity
  services.gnome.games.enable = false;
  services.gnome.core-utilities.enable = false;
  environment.gnome.excludePackages =
    (with pkgs; [ gnome-tour gnome-shell-extensions ]);
  programs.gnome-terminal.enable = false; # make sure you have another terminal program ready before you set this to false
}

And you can install specific utilities you want through home.packages, so you can keep your system as lean as possible.

{ inputs, lib, config, pkgs, ... }: {
  # for example, we want to have file manager and secret manager back
  home.packages = with pkgs; [
    nautilus
    seahorse
  ];
}

Customizing Gnome settings with `dconf`

dconf is the low level key-based configuration system used by Gnome, and we can interact with it with Home Manager and create a reproducible configuration of Gnome. To define dconf settings with Home Manager, you can use dconf.settings.

Personally I found that dconf-editor very useful for exploring settings and finding keys in dconf.

{ inputs, lib, config, pkgs, ... }: {

  dconf.settings = {
    "org/gnome/settings-daemon/plugins/power" = {
      power-button-action = "interactive";
      sleep-inactive-ac-timeout = 0;
      sleep-inactive-ac-type = "nothing";
      sleep-inactive-battery-type = "nothing";
    };
    "org/gnome/desktop/datetime" = { automatic-timezone = true; };
  };
}

Customizing fonts settings in Gnome

To set system font families, you can set the value of keys org/gnome/desktop/interface/monospace-font-name and org/gnome/desktop/interface/document-font-name in dconf through dconf.settings. Make sure you have installed those font packages through home.packages.

{ inputs, lib, config, pkgs, ... }: {
  home.packages = with pkgs; [
    (nerdfonts.override { fonts = [ "0xProto" ]; }) # avoid installing everything from nerdfonts
    Inter
    noto-fonts-cjk-sans
    noto-fonts-cjk-serif
  ];

  dconf.settings = {
    "org/gnome/desktop/interface" = {
      monospace-font-name = "0xProto Nerd Font Mono";
      document-font-name = "Inter";
    };
  };
}

Customizing default font for applications

As Gnome respects configuration of fontconfig, you can use fonts.fontconfig for defining default font that will be used by applications(e.g. Firefox). You can define multiple fonts as default at fonts.fontconfig.defaultFonts.monospace, fonts.fontconfig.defaultFonts.sansSerif, fonts.fontconfig.defaultFonts.serif and fonts.fontconfig.defaultFonts.emoji.

Some applications respect gtk configuration instead of fontconfig, to set that we can define the font we want to use in gtk.font.

{ inputs, lib, config, pkgs, ... }: {
  # omitted for brevity
  fonts.fontconfig = {
    enable = true;
    defaultFonts = {
      monospace = [ "0xProto Nerd Font Mono" "Noto Sans Mono CJK HK" ];
      serif = [ "Noto Serif CJK HK" ];
      sansSerif = [ "Inter" "Noto Sans CJK HK" ];
    };
  };

  gtk = {
    enable = true;
    font = {
      name = "Inter";
      package = pkgs.inter;
    };
  };
}

Enable font-hinting and font-antialiasing

If you have a good monitor, you might notice that by default the rendering of font in Gnome is blurry and not as crisp as other systems. To enable RGBA font antialiasing and full font hinting, you can set org/gnome/desktop/interface/font-hinting = "full" and org/gnome/desktop/interface/font-antialiasing = "rgba" through dconf.settings.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/desktop/interface" = {
      font-hinting = "full";
      font-antialiasing = "rgba";
    };
  };
}

Increase system font size

You can set the desired system font size alongside the name of font at org/gnome/desktop/interface/font-name. The name of font should match org/gnome/desktop/interface/document-font-name.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/desktop/interface" = {
      font-name = "Inter 11";
    };
  };
}

Since my CPU does not come with an integrated GPU, I cannot put my Nvidia GPU into offload mode during suspend. After waking it up, sometimes text on my screen with be broken. If I set NVreg_PreserveVideoMemoryAllocations=1 kernel parameter to preserve video memory after suspend, I can’t even enter suspend.

Therefore, I decided to disable suspend completely. After searching up documentation for a few hours, I finally figured out that can only be disabled through systemd configuration. No Gnome extension needed.

{ config, pkgs, lib, inputs, ... }:

{
  systemd.targets.sleep.enable = false;
  systemd.targets.suspend.enable = false;
  systemd.targets.hibernate.enable = false;
  systemd.targets.hybrid-sleep.enable = false;
}

Disabling auto suspend

By default, Gnome will automatically put the computer into suspend after a while. To disable that behavior, you can set org/gnome/settings-daemon/plugins/power/sleep-inactive-ac-timeout = 0 in dconf through dconf.settings.

If you want to selectively enable auto suspend based on your power supply, org/gnome/settings-daemon/plugins/power/sleep-inactive-ac-type and org/gnome/settings-daemon/plugins/power/sleep-inactive-battery-type might be useful to you.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/settings-daemon/plugins/power" = {
      power-button-action = "interactive";
      sleep-inactive-ac-timeout = 0;
      sleep-inactive-ac-type = "nothing";
      sleep-inactive-battery-type = "nothing";
    };
  };
}

Customizing mouse speed and profile

If you want to game on Gnome, mouse acceleration is definitely something you want to disable. To tweak mouse speed and profile, you can set org/gnome/desktop/peripherals/mouse/accel-profile and org/gnome/desktop/peripherals/mouse/speed in dconf through dconf.settings.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/desktop/peripherals/mouse" = {
      accel-profile = "flat";
      speed = 0.5;
    };
  };
}

Customizing wallpapers

If you want to set your own wallpapers and change them based on light and dark mode, you can set org/gnome/desktop/background/picture-uri and org/gnome/desktop/background/picture-uri-dark in dconf through dconf.settings.

{ inputs, lib, config, pkgs, ... }: {
  dconf.settings = {
    "org/gnome/desktop/background" = {
      picture-uri =
        "file:///home/foobar/whereever-you-store-your-wallpapers/example.jpg";
      picture-uri-dark =
        "file:///home/foobar/whereever-you-store-your-wallpapers/example-dark.jpg";
    };
  };
}

Reduce input latency for gaming

You can enable variable refresh rate (VRR) through dconf.settings to reduce input latency. This experimental feature is only available since Gnome version 46.

Customizing Firefox with Nix and Home Manager

Sun, 05 Jan 2025 00:00:00 GMT

In this post I will share my experience on customizing Firefox using Nix and Home Manager, so you can easily create reproducible and declarative configuration to manage your Firefox.

Installing Firefox with Home Manager

Installing Firefox with Home Manager is easy on Linux, just set programs.firefox.enable = true and you are good to go.

{ inputs, lib, config, pkgs, ... }: {
  programs.firefox = {
    enable = true;
  };
}

However, if you are using Home Manager on MacOS, you will find out that the firefox package is broken on darwin. Luckily, there is a nixpkgs-firefox-darwin overlay that provides a firefox derivation that works on darwin. Browsers derived from firefox, such as librewolf and floorp are also supported by this overlay.

To use this overlay, add this flake as an input to your flake, and pass the nixpkgs-firefox-darwin.overlay to overlays in the import statement of nixpkgs.

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/release-24.11";

    home-manager.url = "github:nix-community/home-manager/release-24.11";
    home-manager.inputs.nixpkgs.follows = "nixpkgs";

    nixpkgs-firefox-darwin.url =
      "github:bandithedoge/nixpkgs-firefox-darwin";
    nixpkgs-firefox-darwin.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, home-manager, nixpkgs-firefox-darwin, ... }@inputs:
    let
      inherit (self) outputs;
      lib = nixpkgs.lib;
      darwinArmSystem = "aarch64-darwin";

      darwinArmPkgs = import nixpkgs {
        system = darwinArmSystem;
        overlays = [
          nixpkgs-firefox-darwin.overlay # add the overlay from the input
        ];
      };
    in {
      homeConfigurations = {
        "darwin" = home-manager.lib.homeManagerConfiguration {
          pkgs = darwinArmPkgs;
          modules = [ ./home-manager/firefox.nix ];
        };
      };
    };
}

Configuring Firefox with Home Manager

To configure options avaliable to Firefox in about:config editor with Home Manager, you can create a profile with programs.firefox.profiles.<name> and define options specific to this profile at programs.firefox.profiles.<name>.settings.

{ inputs, lib, config, pkgs, ... }: {
  programs.firefox = {
    enable = true;
    profiles = {
      admin = {
        id = 0;
        name = "admin";
        isDefault = true;
        settings = {
          "extensions.pocket.enabled" = false;
          "browser.toolbarbuttons.introduced.pocket-button" = false;
        };
      };
    };
  };
}

I have tried to find an exhaustive list for all options in about:config, but based on this conversation that seems to be non-existent.

Installing Firefox extensions with Home Manager

Firefox extensions are not packaged in the official nixpkgs repository but in the nur repository. To use this repository, you have to include it as an input for a flake, and apply an overlay in the import statement of nixpkgs.

{
  inputs = {
    # omitted for brevity

    nur.url = "github:nix-community/NUR/master";
    nur.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, home-manager, nixpkgs-firefox-darwin, nur, ... }@inputs:
    let
      inherit (self) outputs;
      lib = nixpkgs.lib;
      darwinArmSystem = "aarch64-darwin";

      darwinArmPkgs = import nixpkgs {
        system = darwinArmSystem;
        overlays = [
          nixpkgs-firefox-darwin.overlay
          nur.overlays.default # add the overlay from the input
        ];
      };
    in {
      # omitted for brevity
    };
}

After applying the overlay, we can then include Firefox extensions as packages at programs.firefox.profiles.<name>.extensions.

{ inputs, lib, config, pkgs, ... }: {
  programs.firefox = {
    enable = true;
    profiles = {
      admin = {
        id = 0;
        name = "admin";
        isDefault = true;
        extensions = with pkgs; [
          # installing bitwarden and ublock-origin through nur
          nur.repos.rycee.firefox-addons.bitwarden
          nur.repos.rycee.firefox-addons.ublock-origin
        ];
      };
    };
  };
}

When I was configuring my own flake, after doing the above, I thought I have done everything I need to set up extensions in Firefox. However, I found out that programs.firefox.profiles.<name>.extensions would only install but not enable the extensions for you.

Enable Firefox extensions with `policies.json`

Firefox allows users to customize its functionality with policies.json. Skimming through the available options, I notice that we can force an extension to be enabled with Extensions.Locked. This unobvious option is exactly what we need.

Extensions.Locked is an array that accepts the ID of a Firefox extension. You can find out the ID of your extensions in about:debugging#/runtime/this-firefox.

Another useful option is ExtensionSettings, which allow you to customize where to show the extension, for example pinning it to the navbar. There are a lot more you can do with policies.json, I highly recommend you go through the documentation.

The corresponding Home Manager option to manage policies.json is programs.firefox.policies.

{ inputs, lib, config, pkgs, ... }: {
  programs.firefox = {
    enable = true;
    policies = {
      Extensions = {
        Locked = [
          # enable both extensions
          "uBlock0@raymondhill.net"
          "446900e4-71c2-419f-a6a7-df9c091e268b" # Extension ID for bitwarden
        ];
      };
      ExtensionSettings = {
        # pin both extensions in the navbar
        "uBlock0@raymondhill.net" = { default_area = "navbar"; };
        "446900e4-71c2-419f-a6a7-df9c091e268b" = { default_area = "navbar"; };
      };
    };
    # omitted for brevity
  };
}

Defining custom search engines and its shortcut alias

To define custom search engines and its shortcut alias in Firefox with Home Manager, you can define them at programs.firefox.profiles.<name>.search.engines. The following is my definition that allows me to search packages in https://search.nixos.org/packages after typing a shortcut alias of @nix.

{ inputs, lib, config, pkgs, system, isDarwin, ... }: {
  programs.firefox = {
    # omitted for brevity
    profiles = {
      admin = {
        search = {
          engines = {
            "Nix Packages" = {
              urls = [{
                template = "https://search.nixos.org/packages";
                params = [
                  {
                    name = "type";
                    value = "packages";
                  }
                  {
                    name = "query";
                    value = "{searchTerms}";
                  }
                ];
              }];

              icon =
                "${pkgs.nixos-icons}/share/icons/hicolor/scalable/apps/nix-snowflake.svg";
              definedAliases = [ "@nix" ];
            };
          };
        };
      };
    };
  };
}

Dockerize SvelteKit with adapter-static and Nginx

Fri, 03 Jan 2025 00:00:00 GMT

I have encountered a few unobvious pitfalls when I create the Dockerfile for this blog, which is a SvelteKit project built with @sveltejs/adapter-static. For the sake of contributing back to the community, I would like to share those pitfalls and some tips with you on building a Docker image that follows best practises.

Build SvelteKit in Docker

To reduce the size of the Docker image as much as possible, only the output of building SvelteKit should be included in the image. With a multi-stage build, only the final layer will be included in the image, and the rest will be discarded. By doing so, we can discard all dependencies of the building process(e.g. node_modules), keeping the image slim.

FROM node:22.12.0-slim as base
WORKDIR /usr/src/app

FROM base AS install
RUN mkdir -p /temp/dev
COPY package.json package-lock.json /temp/dev/
RUN cd /temp/dev && npm install --frozen-lockfile

RUN mkdir -p /temp/prod
COPY package.json package-lock.json /temp/prod/
RUN cd /temp/prod && npm install --frozen-lockfile --production

FROM base AS prerelease
COPY --from=install /temp/dev/node_modules node_modules
COPY . .

ENV NODE_ENV=production
RUN npm run build

However, this Docker image would not run and response to request yet. With @sveltejs/adapter-static, the output of SvelteKit is only a set of static files, and we need to run a file server to serve them. I am going to pick nginx, a popular and performant file server to serve these files.

As I run my Docker images in Kubernetes, I am relatively sensitive with what privileges I need to give my containers. Most people out there would probably use nginx in their Docker image, but this image would run nginx as root, and by default listen to port 80, which is a priviledged port. Running a file server shouldn’t require root permission at all. A better option is to use nginxinc/nginx-unprivileged, which would run nginx as non-root and listen to port 8080 by default.

# omitted for brevity
FROM nginxinc/nginx-unprivileged:1.26.2-alpine AS release
COPY --from=prerelease --chown=nginx:nginx /usr/src/app/build /usr/share/nginx/html
EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

Now if you build and run your Docker image, you should be able to serve your SvelteKit’s output.

Handling directories without `index.html`

Unless you have set trailingSlash = 'always' in your SvelteKit project, you will notice that there is only one index.html in your build output. Pages with a nested path, say /post/my-post-title, would be generated as build/post/my-post-title.html instead of build/post/my-post-title/index.html. nginx will refuse serving these routes with its default configuration. To fix that, we have to provide our own configuration of nginx, through the Dockerfile.

First of all, create an nginx.conf at the same level as your Dockerfile.

server {
    listen 8080;
    listen [::]:8080;
    root /usr/share/nginx/html;
    server_name _;

    location / {
        try_files $uri $uri.html; # handle the output of SvelteKit correctly, by using a fallback pattern.
    }

    include /etc/nginx/mime.types; # handle mime.
}

And then copy it and overwrite the default configuration file of nginx.

FROM nginxinc/nginx-unprivileged:1.26.2-alpine AS release
COPY --from=prerelease --chown=nginx:nginx /usr/src/app/build /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf # [!code ++]
EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

Handling 404 error

Since we are using @sveltejs/adapter-static, all the HTTP errors are handled during prerendering, with the exception of 404. A simple way to handle this is to update nginx.conf and use the internal error page of nginx.

server {
    # omitted for brevity
    location / {
        try_files $uri $uri.html $uri/ =404; # if we cannot find a match, route to 404.
    }


    error_page 404 /404.html; # if 404, route the request to 404.html.


    # return the default error page of nginx.
    location = /404.html {
        internal;
    }
}

Even though SvelteKit would render +error.svelte for 404 during development(i.e. vite dev), it does not generate that page for you automatically in build time. To generate that, you have to specify the fallback in the adapter’s arguments.

import adapter from '@sveltejs/adapter-static';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  // omitted for brevity
  kit: {
    adapter: adapter({

      fallback: '404.html' // this filename can be whatever you want
    }),
  },
};

export default config;

Then when you run vite build, you should be able to find a 404.html in your build directory. To make nginx route to this page whenever a 404 is encountered, you simply need to remove the route to the internal error page.

server {
    # omitted for brevity

    location = /404.html {
      internal;
    }
}

Serving precompressed assets

To improve performance of your site, you should served your assets with compression, such as with gzip or brotli. @sveltejs/adapter-static supports precompressing, creating compressed assets for you in the build time.

import adapter from '@sveltejs/adapter-static';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  // omitted for brevity
  kit: {
    adapter: adapter({
      precompress: true
    }),
  },
};

export default config;

After precompressing your assets, you have to make nginx serve these assets. Setting gzip_static will allow nginx to serve gzip files.

server {
    # omitted for brevity
    gzip_static on;
}

If you want to use brotli instead for better compression, you will need to instsall ngx_brotli module and set brotli_static. I didn’t go that far, but I found a Docker image with this module installed.

How to remove old NixOS generations in your system and bootloader

Wed, 01 Jan 2025 00:00:00 GMT

I remembered running around 60 times of nixos-rebuild switch to make all my hardware and Wi-Fi works in NixOS, in the beginning of my dual booting NixOS and Windows adventure. The next time when I boot, I saw all those old and imperfect generations and I had to press the Down button for 3 seconds until I reach the Windows 11 boot options… Those generations have polluted my bootloader menu and took over my disk space, which is a pain in the back side.

In this Nix tips, I am going to show you a few tricks to remove and avoid keeping old NixOS generations in your system and bootloader menu.

Limit the number of generations kept by NixOS

There is a NixOS configuration called boot.loader.systemd-boot.configurationLimit for limiting the maximum number of generations to keep in your system. When the generations you have exceeded the limit, the oldest one will be removed from your boot menu and marked for garbage collection.

{ config, pkgs, lib, ... }:

{
    boot.loader.systemd-boot.configurationLimit = 10; # only 10 generations are kept
}

Use `nixos-rebuild test` and `nixos-rebuild dry-activate` for testing your configuration

nixos-rebuild switch should be the first command you came across when you start your NixOS, but you should not abuse this command for just experimenting configuration. By itself, this command would build, activate and store a generation in the bootloader menu. Using this for testing, you will easily pollute your bootloader menu.

Unlike nixos-rebuild switch, nixos-rebuild test would only build and activate a new generation based on your current configuration, and it would not add this generation to the bootloader menu. This is really useful if you just want to try out a new configuration.

# if you are not using Nix Flake to manage your NixOS configuration
nixos-rebuild test
# if you are using Nix Flake to manage your NixOS configuration
nixos-rebuild test --flake .#replaceMeWithYourSystemName --use-remote-sudo

If you just want to verify if your current configuration would build without error, you can use nixos-rebuild dry-activate. This command would only build your configuration, but it would not activate or store it in the bootloader menu. On top of that, it would show what changes would be made comparing with the current one, if this generation is activated.

# if you are not using Nix Flake to manage your NixOS configuration
nixos-rebuild dry-activate
# if you are using Nix Flake to manage your NixOS configuration
nixos-rebuild dry-activate --flake .#replaceMeWithYourSystemName --use-remote-sudo

I highly recommend you to check out the doc of nixos-rebuild, and discover other available arguments.

Periodically clean up old generations

NixOS provides an option called nix.gc, allowing you to set up a systemd service to automatically garbage collect old generations for you.

{ config, pkgs, lib, ... }:

{
  nix.gc = {
    automatic = true;
    dates = "*-*-* 21:00:00"; # conduct a GC operation every night at 2100
    options = "--delete-older-than 7d";
  };
}

If you can’t wait and want to get rid of all generations right away, except the latest one, you can use nix-collect-garbage and nixos-rebuild boot.

Remember you have to run nix-collect-garbage with sudo or as root, otherwise the old generations would not be removed, yet the command would still exit without error.

sudo nix-collect-garbage -d
# if you are not using Nix Flake to manage your NixOS configuration
nixos-rebuild boot
# if you are using Nix Flake to manage your NixOS configuration
nixos-rebuild boot --flake .#replaceMeWithYourSystemName --use-remote-sudo