Category: Tools & Setup

Tools & Setup is where orthogonal.info curates practical, battle-tested guides on developer productivity tools, CLI utilities, self-hosted software, and environment configuration. Whether you are bootstrapping a new development machine, evaluating self-hosted alternatives to SaaS products, or fine-tuning your terminal workflow, this category delivers step-by-step walkthroughs grounded in real-world experience. Every article is written with one goal: help you build a faster, more reliable, and more enjoyable development environment.

With over 25 in-depth posts and growing, Tools & Setup is one of the most active categories on the site — reflecting just how much time engineers spend (and save) by getting their tooling right from day one.

Key Topics Covered

Command-line productivity — Shell customization (Zsh, Fish, Starship), terminal multiplexers (tmux, Zellij), and CLI utilities like ripgrep, fd, fzf, and bat that supercharge daily workflows.
Self-hosted alternatives — Deploying and configuring tools like Gitea, Nextcloud, Vaultwarden, and Uptime Kuma so you own your data without sacrificing usability.
IDE and editor setup — Configuration guides for VS Code, Neovim, and JetBrains IDEs, including extension recommendations, keybindings, and remote development workflows.
Development environment automation — Using Ansible, Homebrew, Nix, dotfiles repositories, and container-based dev environments (Dev Containers, Devbox) to make setups reproducible.
Git workflows and tooling — Advanced Git techniques, hooks, aliases, and GUI clients that streamline version control for solo developers and teams alike.
API testing and debugging — Hands-on guides for curl, HTTPie, Postman, and browser DevTools to debug REST and GraphQL APIs efficiently.
Package and runtime management — Managing multiple language runtimes with asdf, mise, nvm, and pyenv, plus dependency management best practices.

Who This Content Is For
This category is designed for software engineers, DevOps practitioners, system administrators, and hobbyist developers who want to work smarter, not harder. Whether you are a junior developer setting up your first Linux workstation or a senior engineer optimizing a multi-machine workflow, you will find actionable advice that respects your time. The guides assume basic command-line comfort but explain advanced concepts clearly.

What You Will Learn
By exploring the articles in Tools & Setup, you will learn how to automate repetitive environment tasks so a fresh machine is productive in minutes, not days. You will discover modern CLI replacements for legacy Unix tools, understand how to evaluate self-hosted software against its SaaS equivalent, and gain confidence configuring complex development stacks. Each guide includes copy-paste commands, configuration snippets, and links to upstream documentation so you can adapt the advice to your own infrastructure.

Start browsing below to find your next productivity upgrade.

  • SSH Server Hardening in 2026: A Version-Aware sshd_config Guide

    An SSH configuration can pass sshd -t and still lock out every administrator. Syntax validation cannot prove that your key works, that your account belongs to an allowed group, or that PAM will not offer another password path. Safe SSH server hardening starts with the effective configuration, changes only the controls you need, and keeps a tested recovery path open until a new login succeeds.

    This guide targets current OpenSSH releases while calling out settings that vary by version or Linux distribution. Do not paste the whole block into a remote server and reload it blindly. Read the effective values first, test each authentication path, and keep provider console or physical-console access available.

    Read the Effective Configuration First

    /etc/ssh/sshd_config is not always the whole configuration. Distributions may load files through Include, package updates may change defaults, and Match blocks can alter values for a user or source address. The upstream default is only a reference; the output from your server is what matters.

    sudo sshd -t
    sudo sshd -T | grep -E \
    '^(permitrootlogin|pubkeyauthentication|passwordauthentication|kbdinteractiveauthentication|authenticationmethods|authorizedkeysfile|allowgroups|allowusers|ciphers|kexalgorithms|macs|channeltimeout) '
    

    sshd -t checks configuration syntax and host-key sanity. sshd -T prints the effective server configuration. OpenSSH generally uses the first obtained value for each keyword, so an early vendor or cloud-image drop-in may win over a later edit in the main file. Inspect the Include order and files such as sshd_config.d/50-cloud-init.conf. If you use Match rules, add -C with representative connection values. The official sshd_config manual documents those parameters.

    One common misconception is that root password login is enabled by default everywhere. Current upstream OpenSSH defaults PermitRootLogin to prohibit-password, which blocks password and keyboard-interactive authentication for root but still permits public-key login. A distribution or existing configuration can override that value, so check sshd -T rather than guessing.

    Configure Key-Only Authentication Without a PAM Back Door

    For a server that should accept public keys only and does not use keyboard-interactive MFA, this is a clear baseline:

    PubkeyAuthentication yes
    PasswordAuthentication no
    KbdInteractiveAuthentication no
    AuthenticationMethods publickey
    PermitRootLogin no
    

    PasswordAuthentication no disables the SSH password method. It does not, by itself, prove that a PAM-backed keyboard-interactive prompt cannot accept a password. Disabling KbdInteractiveAuthentication closes that second interactive path, while AuthenticationMethods publickey makes the intended policy explicit.

    Do not copy this block if the server intentionally uses Duo, an OATH token, or another PAM-backed second factor. That design normally needs KbdInteractiveAuthentication yes and an authentication chain such as publickey,keyboard-interactive:pam. The PAM stack must then be audited so it accepts the intended factor rather than silently restoring password login. Also avoid changing UsePAM casually; distributions use PAM for account and session controls in addition to authentication.

    Prefer a named administrative account with sudo over direct root login. If an automation key genuinely requires root, PermitRootLogin prohibit-password can retain key access, but restrict that key with an effective forced command and review its lifecycle.

    Restrict Accounts Before Tuning Cryptography

    An allowlist removes SSH access from application and service accounts that never need a shell:

    AllowGroups sshusers
    MaxAuthTries 4
    LoginGraceTime 30
    LogLevel VERBOSE
    

    Create the group and add every required administrator before enabling AllowGroups. Test automation and emergency accounts too. Each public key offered by an agent can consume an authentication attempt, so test MaxAuthTries 4 with real clients; use client-side IdentitiesOnly yes per host when an agent holds many keys. LoginGraceTime 30 limits how long an unauthenticated connection may wait.

    LogLevel VERBOSE can add the public-key fingerprint used during authentication, which is useful when keys have owners and expiry dates. Check the logs on your distribution before relying on this: logging destinations and the detail already present at the default level vary by OpenSSH build and system logger.

    Changing port 22 may reduce automated log noise, but it is not an access-control boundary. If you move the port, update the host firewall, cloud firewall, NAT rules, monitoring, configuration management, and any SELinux port policy before reloading SSH. On Ubuntu systems where ssh.socket is active, the socket unit controls the listener and changing Port alone may not move it. A firewall allowlist or VPN is a stronger control than a surprising port number.

    Do Not Freeze a Modern Server to an Old Algorithm List

    Hard-coding complete Ciphers, KexAlgorithms, and MACs lists often makes a current server worse. A replacement list discards the vendor defaults, including algorithms added in newer releases. The current upstream proposal list includes ChaCha20-Poly1305, modern AES modes, and hybrid post-quantum key exchange.

    Inspect what the installed versions support and what the daemon will actually offer:

    ssh -Q cipher
    ssh -Q kex
    ssh -Q mac
    sudo sshd -T | grep -E '^(ciphers|kexalgorithms|macs) '
    

    Keep a patched OpenSSH release and its distribution crypto policy unless you have a documented compliance requirement. If one legacy algorithm must be removed or one compatibility algorithm temporarily added, OpenSSH supports list modifiers such as -, +, and ^. Make the smallest version-tested change instead of replacing the entire set. The OpenSSH release notes show why static lists age badly.

    Dead-Client Detection Is Not an Idle-Session Timeout

    ClientAliveInterval and ClientAliveCountMax detect an unresponsive client. A user who is idle at a shell still has a responsive SSH client, which answers the encrypted keepalive message and remains connected. Those settings therefore do not enforce a ten-minute inactivity policy.

    OpenSSH 9.2 and later can use ChannelTimeout to expire an inactive session channel:

    ChannelTimeout session=10m
    

    This applies to session channels, including interactive shells, commands, SCP, and SFTP. It can terminate legitimate quiet work, and closing a channel does not guarantee that every child resource disappears. Treat it as an explicit policy choice, test it with the installed OpenSSH version, and keep it out of shared baselines for hosts that run long quiet commands. Older releases will reject the directive; sshd -t is the compatibility check. Later, OpenSSH 9.7 added a global timeout type for connections with several channel types.

    Audit Every Authorized-Key Source

    Do not assume every home directory is under /home. Root, service accounts, directory-backed users, and custom home paths will be missed. First inspect AuthorizedKeysFile and AuthorizedKeysCommand in the effective configuration. The upstream file default is .ssh/authorized_keys .ssh/authorized_keys2, relative to each user’s actual home directory.

    For the default relative paths, this shell loop reports fingerprints for files that exist across accounts returned by getent:

    getent passwd |
    while IFS=: read -r user _ uid gid gecos home shell; do
      for rel in .ssh/authorized_keys .ssh/authorized_keys2; do
        file="$home/$rel"
        [ -f "$file" ] || continue
        printf '\n%s\t%s\n' "$user" "$file"
        sudo ssh-keygen -lf "$file"
      done
    done
    

    If AuthorizedKeysFile contains absolute paths or tokens such as %u and %h, audit those resolved locations instead. If AuthorizedKeysCommand is configured, review that program, its dedicated execution user, and the external key source. Removing a file does not revoke a key supplied by a command or trusted user CA.

    For hardware-backed keys, the related YubiKey SSH authentication guide covers OpenSSH security-key key types. The secure homelab remote-access guide covers the network layer around SSH.

    Apply Changes Without Locking Yourself Out

    1. Confirm that an out-of-band console or physical console works. A theoretical recovery path is not enough.
    2. Keep the current SSH session open and verify public-key login in a second session.
    3. Back up the active configuration and change the smallest possible set of directives.
    4. Run sudo sshd -t, then inspect the relevant values with sudo sshd -T.
    5. Reload rather than stop the daemon. The unit is commonly sshd on RHEL-family systems and ssh on Debian-family systems; use the installed name. If socket activation controls a changed port, update and restart the socket unit too.
    6. Open a third, completely new connection. Test the allowed key, a disallowed account, and a forced password-only attempt.
    7. Close the backup session only after every expected success and failure behaves correctly.

    A syntax check cannot detect a missing group membership, a wrong key, a firewall mistake, or an unintended PAM path. That is why a new connection and working console access are part of the change, not optional cleanup afterward.

    References and Useful Hardware

    Start with the current sshd_config manual and sshd manual, then compare them with your distribution’s package documentation. For deeper reading and hardware-backed access, these are practical options:

    Full disclosure: the Amazon links below are affiliate links. I may earn a commission from qualifying purchases at no extra cost to you.

    For more security field notes and practical infrastructure checks, follow the author’s Alpha Signal channel on Telegram.

  • Browser Fingerprinting Defenses That Actually Work in 2026

    I ran the EFF’s Cover Your Tracks tool against three different browsers on the same machine last month, expecting the privacy-focused one to come out clean. It didn’t. The browser I had configured with the most protection settings enabled registered as more uniquely identifiable than a stock Chrome install, because the unusual combination of protections itself formed a distinctive fingerprint. That experience pushed me to spend a few weeks actually understanding how browser fingerprinting works and which defenses hold up in practice.

    This is not a survey of every privacy tool on the market. It’s field notes on what I tested, what worked, and where the tradeoffs landed.

    What Browser Fingerprinting Actually Collects

    Browser fingerprinting assembles a profile from dozens of data points that your browser leaks during normal page loads. The obvious ones: user-agent string, screen resolution, timezone, and installed fonts. The less obvious: how your GPU renders specific WebGL test scenes, the timing characteristics of your CPU doing JavaScript math, the exact set of audio codecs your browser supports, and whether your canvas element draws anti-aliased edges identically to other users running the same browser version on different hardware.

    Each data point on its own is close to meaningless. Combined across 30-50 signals, they frequently produce a unique identifier. The EFF’s Cover Your Tracks project, which tests real-world fingerprinting against a database of millions of browser profiles, found that the majority of browsers they tested were uniquely identifiable—or nearly so—without any cookies at all.

    The key difference from cookie tracking: there’s nothing to delete (the same reason EXIF metadata leaks matter). Clearing your browser history, blocking third-party cookies, and using incognito mode all leave your fingerprint intact. The fingerprint is based on the state of your hardware and software, not on stored data.

    Testing Your Own Browser Fingerprint

    Before changing anything, it’s worth establishing a baseline. The most useful free tool is Cover Your Tracks from EFF. It tests your browser against a live database and tells you whether your fingerprint is unique, gives you a protection rating, and shows you which signals are contributing most to your identifiability.

    The second tool worth running is BrowserLeaks, which breaks out individual API leaks—WebGL renderer info, canvas fingerprint, AudioContext fingerprint, font enumeration—so you can see exactly which vectors are exposing the most identifying information.

    A few things to note when interpreting results:

    • A “unique” result doesn’t mean someone is currently tracking you—it means you could be tracked stably if any site embeds a fingerprinting script.
    • Results change with browser updates, OS updates, and hardware changes, so fingerprints have some natural decay.
    • Testing with a VPN active tests the VPN’s IP reputation but doesn’t change your browser fingerprint at all. Those are separate threat models.

    Which Defenses Work and Which Backfire

    Tor Browser remains the most technically effective solution. It normalizes the fingerprint across all Tor Browser users by using the same default window size, the same font set, the same rendering behavior, and by funneling all traffic through the Tor network. The cost is performance and broken functionality on sites that block Tor exit nodes. For high-stakes privacy work—journalists, activists, researchers accessing sensitive materials—this is the correct tool.

    Firefox with privacy.resistFingerprinting enabled is the most practical daily-driver option. The privacy.resistFingerprinting flag in about:config enables a suite of protections that spoof or normalize many fingerprinting signals: it reports a fixed screen size regardless of your actual display, randomizes canvas output per session, and limits timezone leakage. Combined with uBlock Origin and Firefox’s Enhanced Tracking Protection set to Strict, this produces a substantially more private profile than default Chrome.

    The caveat I encountered: layering too many additional protections on top of this actually increases uniqueness. A Firefox install with privacy.resistFingerprinting, plus a custom user.js with 40 additional flags, plus a fingerprint-spoofing extension, registers as more unique than Firefox with just the one flag enabled, because the combination is rare. The EFF’s fingerprinting research explicitly documents this paradox—being in a crowd of users with identical settings is the goal, not having the most aggressively customized setup.

    Brave Browser takes a different approach: randomizing fingerprint signals per session rather than normalizing them. Instead of making your canvas output match everyone else’s, Brave makes your canvas output slightly different on every page load and every session. This breaks the cross-site tracking use case but doesn’t make you blend into a crowd the same way Tor Browser does.

    Chrome with third-party cookies blocked is better than unmodified Chrome but still highly fingerprintable. Adding uBlock Origin and keeping the browser updated provides more benefit than any fingerprint-specific settings.

    Practical Setup for Daily Use

    After testing, my working setup uses Firefox as the primary browser with the following configuration:

    • privacy.resistFingerprinting = true in about:config
    • Enhanced Tracking Protection set to Strict
    • uBlock Origin in medium mode (blocks third-party scripts by default, whitelist per site as needed)
    • DNS-over-HTTPS enabled, pointing to a resolver that doesn’t log queries
    • Firefox containers for site isolation—financial sites, social sites, and general browsing each get separate cookie jars

    That’s five changes, none requiring technical expertise beyond reading a settings page. The result when I re-ran Cover Your Tracks: no longer uniquely identified, fingerprint protected against tracking.

    I keep a second browser—Chromium—for sites that break with strict tracking protection. Compartmentalizing rather than fighting every site is more sustainable.

    For people running their own servers or homelabs, a Pi-hole DNS sinkhole handles fingerprinting-adjacent ad network domains at the network level and protects all devices including phones and smart TVs that can’t run browser extensions. The post on homelab hardware for self-hosted services covers the hardware side of running always-on infrastructure like Pi-hole alongside other services.

    Mobile: A Different and Harder Problem

    Mobile browsers are a significantly weaker privacy environment. Safari on iOS has Intelligent Tracking Prevention, which limits cross-site cookie tracking but doesn’t address fingerprinting. Brave on Android provides the same randomization approach as the desktop version. Firefox Focus blocks trackers aggressively but lacks the extension ecosystem of desktop Firefox.

    The harder problem on mobile is the app layer. Browser fingerprinting defenses don’t apply to native apps, which can collect device identifiers, precise GPS coordinates, and behavioral data through SDKs embedded in apps you use for entirely unrelated purposes. The most effective mobile privacy measure is auditing what apps have network permissions and deleting anything where the data collection isn’t worth the functionality. iOS’s App Privacy Report and Android’s permission manager both make this tractable now.

    For security-focused reading that covers both browser and application-layer tracking, the books below cover the technical and policy dimensions. These are affiliate links, which means I may earn a small commission from qualifying purchases at no extra cost to you.

    What to Actually Do This Weekend

    If you’ve read this far and want a concrete action rather than another things-to-consider list:

    1. Open coveryourtracks.eff.org in your current browser and run the test. Note your result.
    2. If you’re on Firefox, enable privacy.resistFingerprinting and install uBlock Origin if you haven’t. Re-run the test.
    3. If you’re on Chrome, consider switching to Firefox or Brave for personal browsing. Keep Chrome for work compatibility if needed.
    4. Check your phone’s App Privacy Report (iOS) or app permissions (Android) and revoke network access from apps that don’t need it.

    The goal isn’t perfect anonymity—that requires operational discipline that most people don’t need in their daily lives. The goal is not being the easiest target in the room. Reducing your fingerprint’s uniqueness removes you from the highly-identifiable population that advertising networks, data brokers, and tracking scripts can follow reliably across sessions and sites.

    If you’re running your own infrastructure and want to track how tracking works at the network level, the post on NVMe SSDs and Docker development environments is adjacent reading for people setting up self-hosted DNS and security tooling.

    For more privacy and security field notes, follow Alpha Signal on Telegram.

  • yfinance 1.6.0: Practical Python Stock Data, Live Streaming, and Equity Screening

    I pulled yfinance Python stock data for the first time in 2021, back when the library was still fighting Yahoo’s API changes every few months. In 2026, yfinance 1.6.0 landed on PyPI on August 13th, and the gap between what it could do then and what it can do now is significant enough to revisit. This is not a documentation mirror. These are notes from working with it on a personal portfolio tracker and a couple of backtesting scripts.

    What Changed in Recent Versions Worth Knowing About

    The most practically useful additions in recent yfinance releases are the live data components and the screening API. The WebSocket and AsyncWebSocket classes now expose real-time quote streaming. The Screener and EquityQuery objects let you build structured queries to filter equities without leaving Python. The Market class gives you status and session information for a given exchange. These additions move yfinance closer to being a self-contained data layer for personal projects.

    One important caveat that the PyPI page spells out clearly: Yahoo!, Y!Finance, and Yahoo! finance are registered trademarks of Yahoo, Inc. yfinance is an open-source tool using Yahoo’s publicly available APIs and is not affiliated with or endorsed by Yahoo. Their terms of use explicitly say the API is intended for personal use only. If you are building anything commercial, that is your problem to resolve before writing a single line of code.

    Install is straightforward:

    pip install yfinance

    The default install now includes curl_cffi as a fallback for requests. If you are in an environment where that is a problem — some corporate proxies, older OS images, or constrained containers — the documentation at ranaroussi.github.io/yfinance covers an alternative install path.

    Fetching Historical Data Without Shooting Yourself in the Foot

    The Ticker object is the entry point for single-instrument data. Most people start here and never need anything else.

    import yfinance as yf
    
    msft = yf.Ticker("MSFT")
    
    # Last 3 months of daily data
    hist = msft.history(period="3mo")
    print(hist.tail())
    
    # Specific date range
    hist2 = msft.history(start="2026-01-01", end="2026-08-01")
    print(hist2.shape)
    

    The history() call returns a pandas DataFrame with Open, High, Low, Close, Volume, Dividends, and Stock Splits columns. The index is a timezone-aware DatetimeIndex. That last part trips up new users who try to compare it against naive datetime objects. Always normalize your date comparisons or the filtering will silently return wrong results.

    One thing I missed for longer than I should have: history() auto-adjusts for splits and dividends by default. If you want raw unadjusted prices — for example, when you are cross-referencing a specific broker’s records — pass auto_adjust=False. The default is almost always what you want for portfolio math, but it matters when you are debugging discrepancies with another data source.

    For fetching multiple tickers at once, use yf.download():

    tickers = ["AAPL", "GOOGL", "BRK-B", "VTI"]
    data = yf.download(tickers, period="1y", group_by="ticker")
    

    The resulting DataFrame has a MultiIndex column structure. If you only need closing prices, data["Close"] gives you a clean DataFrame with one column per ticker. That is the shape most backtesting libraries expect.

    Live Streaming and Where It Actually Helps

    The WebSocket class is genuinely new territory for yfinance. For personal projects that want live quote updates without paying for a proper market data feed, it is a workable starting point. Here is a minimal example:

    import yfinance as yf
    
    ws = yf.WebSocket(["AAPL", "MSFT"])
    ws.start()
    
    # Access streaming quotes
    for quote in ws.stream():
        print(quote)
        if some_condition:
            break
    
    ws.stop()
    

    A few practical notes from using this: the streaming data reflects Yahoo’s quote feed latency and hours. During pre-market and post-market, the data you receive may have longer gaps between ticks than during regular session. Do not build hard latency assumptions into logic that processes this stream. Also test the reconnection behavior before depending on it — a dropped connection during the middle of a session should gracefully resume, but verify that on your network before assuming it.

    The AsyncWebSocket version is more suitable if you are integrating this into an async application. If you are running alongside FastAPI, an async queue, or asyncio-based orchestration, the async variant avoids threading headaches.

    Screening Equities with EquityQuery

    The EquityQuery and Screener combination is the most underused feature I have seen people miss. Instead of downloading a universe and filtering in pandas, you can push the filter criteria upstream:

    from yfinance import EquityQuery, Screener
    
    # Stocks with market cap over $10B in technology sector
    q = EquityQuery("and", [
        EquityQuery("gt", ["marketcap", 10_000_000_000]),
        EquityQuery("eq", ["sector", "Technology"])
    ])
    
    screener = Screener()
    results = screener.set_predefined_body(q).fetch()
    print(results["quotes"][:5])
    

    The API mirrors the query structure Yahoo’s screener uses internally. The available fields and their exact names are documented at the yfinance docs site. Some fields behave differently than their names suggest — test on a small query first and validate the shape of the output before building business logic on top of it.

    For backtesting infrastructure, I have found it useful to run a weekly screener pull into a local SQLite database, then do all the historical analysis offline. That pattern keeps you out of rate-limit territory and gives you reproducible inputs for your strategy tests. If you are keeping local databases and want to understand the storage hardware decisions that support that kind of setup, the post on NVMe SSDs for Docker and development workloads covers storage endurance and capacity considerations that apply equally well here.

    What the Library Cannot Do and What to Reach for Instead

    yfinance does not give you tick data, Level 2 order book depth, or historical intraday data beyond what Yahoo’s API exposes. For most personal finance projects and simple backtests, that is fine. If you are doing anything that requires precise intraday execution modeling, you need a real market data vendor.

    The rate-limiting situation is also worth understanding. Yahoo does not publish official rate limits, and the community’s observed behavior varies with region, time of day, and whether you are using a residential or cloud IP. If you are fetching data in bulk — downloading five years of daily history for a thousand symbols — add delays between calls and handle the occasional 429 or empty response gracefully. A simple retry with exponential backoff covers most cases:

    import time
    import yfinance as yf
    
    def fetch_with_retry(ticker, retries=3, delay=2):
        for attempt in range(retries):
            try:
                t = yf.Ticker(ticker)
                data = t.history(period="1y")
                if not data.empty:
                    return data
            except Exception as e:
                print(f"Attempt {attempt+1} failed: {e}")
                time.sleep(delay * (2 ** attempt))
        return None
    

    For a homelab or self-hosted setup where you want to run these scripts on a schedule, having a proper server matters more than the script itself. The homelab hardware guide covers what to look for if you are building out a machine that runs financial data pipelines alongside other services.

    Books are still the most efficient way to build up the conceptual foundation. The Amazon searches below are a useful starting point for the adjacent topics — Python finance programming, algorithmic trading implementations, and the hardware side of running your own infrastructure. These are affiliate links, which means I may earn a commission from qualifying purchases at no extra cost to you.

    A Practical Starting Project

    If you want a concrete thing to build with yfinance rather than just experimenting in a notebook, try a weekly portfolio snapshot script. Once a week, it downloads the last 52 weeks of price history for every position you hold, computes each position’s percentage return against the index of your choice, writes the output to a CSV, and optionally pushes a summary to a Telegram bot or local dashboard.

    That project forces you to handle the MultiIndex DataFrame structure, deal with corporate actions (splits and dividends) correctly, manage missing trading days across different exchanges, and think about where and how you store the output. Those four problems cover most of what you will encounter in more complex work.

    yfinance 1.6.0 is a capable free data layer for personal finance work. It has real limits — it is not a production market data feed, and Yahoo’s terms are personal-use only — but within those limits, the library has grown into something genuinely useful. The screening API and live streaming components in particular are worth building time into if you have been using only the historical data path.

    For more notes on Python tools, financial data workflows, and market research signals, subscribe to Alpha Signal on Telegram.

  • Best NVMe SSDs for Docker Builds: TBW, Capacity, and Cooling

    Best NVMe SSDs for Docker Builds: TBW, Capacity, and Cooling

    The best NVMe SSDs for Docker builds are not necessarily the drives with the largest sequential-read number on the box. Container work creates a mix of layer extraction, package-cache writes, metadata updates, and repeated reads across many small files. Capacity, endurance, cooling, and a sane cache strategy matter alongside peak speed.

    I compared the current Docker storage documentation with manufacturer specifications for three widely available PCIe 4.0 drives. The goal is not to manufacture a benchmark I cannot reproduce on your workstation. It is to show which specifications affect a development workload, which numbers are mostly marketing, and what to measure before spending money.

    Why Docker Builds Put Pressure on Storage

    Docker images are assembled from layers. On current fresh installations, Docker Engine 29 uses the containerd image store by default; upgraded installations may still use the classic overlay2 driver. Docker’s documentation notes that the containerd store keeps both compressed and uncompressed image data, so it can use more disk capacity than the older storage path.

    With overlay2, the first write to a file from a lower image layer can trigger a full-file copy_up into the writable layer. A build also creates and reads package caches, source trees, temporary objects, and image layers. That workload is not represented by a single sequential-read score.

    Before blaming the SSD, check the build itself. Docker recommends ordering layers to protect cache hits, keeping the build context small, and using cache mounts for package managers. A faster drive cannot recover time lost because every source change invalidates a dependency-install layer. See Docker’s official build-cache guidance before buying hardware.

    Start with Capacity, Not the Peak MB/s Number

    Run docker system df and docker buildx du to see how much space images, containers, volumes, and build cache currently consume. Then add the working tree, local databases, virtual machines, and enough free space for firmware garbage collection to work effectively.

    docker system df
    docker buildx du
    docker info --format '{{json .DriverStatus}}'
    

    A 1TB drive can be sufficient for one active codebase. A developer keeping several multi-platform builders, Android toolchains, local model files, and database snapshots may run out of room quickly. Capacity pressure causes manual pruning and makes it tempting to mix irreplaceable source data with disposable caches.

    Docker stores daemon data under /var/lib/docker by default on Linux. The containerd image store uses a separate path for image contents and snapshots, so changing Docker’s data-root does not automatically move that containerd data. Confirm the active backend before designing a dedicated build drive.

    TBW Is More Useful Than a Hero Benchmark

    TBW, or terabytes written, is the manufacturer’s endurance limit used with the warranty period. It is not a prediction that the drive will fail one byte later. It does give you a consistent way to compare capacities within a product family.

    The official specifications rate the 2TB Samsung 990 PRO and 2TB WD_BLACK SN850X at 1,200 TBW. The SN850X data sheet states that its TBW calculation uses the JEDEC client workload and that warranty coverage ends at five years or the endurance limit, whichever comes first. Crucial also lists 1,200 TBW for the 2TB T500.

    Those values are far more informative for a write-heavy development machine than comparing 7,300 MB/s with 7,450 MB/s. Your motherboard generation, thermal limit, filesystem, build graph, and cache-hit rate can dominate that small peak-speed difference.

    Three PCIe 4.0 Choices Worth Comparing

    Samsung 990 PRO: high-end desktop choice

    Samsung lists the 2TB 990 PRO with heatsink at up to 7,450 MB/s sequential read and 6,900 MB/s sequential write. A factory-heatsink version is available, which is useful when the motherboard lacks an M.2 cover or the slot sits near a hot GPU.

    Check the current Samsung 990 PRO 2TB listings on Amazon. Match the exact capacity, heatsink option, and model number rather than trusting a search-result thumbnail.

    WD_BLACK SN850X: TLC and clear endurance specs

    SanDisk’s July 2025 data sheet identifies the SN850X as TLC 3D NAND. The 2TB model is rated for up to 7,300 MB/s sequential read, 6,600 MB/s sequential write, 1,200 TBW, and a five-year limited warranty. Both bare and factory-heatsink versions are listed.

    Check the current WD_BLACK SN850X 2TB listings on Amazon. The heatsink model is taller, so verify clearance in compact systems and laptops.

    Crucial T500: another 2TB, 1,200-TBW option

    Crucial offers the 2TB T500 in versions with and without a heatsink and lists a five-year limited warranty with 1,200 TBW endurance. It belongs on the same shortlist when pricing changes, but compare the exact SKU because heatsink and non-heatsink models are easy to confuse.

    Check the current Crucial T500 2TB listings on Amazon. Do not pay extra for a bundled heatsink if your motherboard already has a suitable M.2 thermal plate.

    Full disclosure: the three Amazon links above are affiliate links. I may earn a commission from qualifying purchases at no extra cost to you.

    Cooling Matters During Repeated Builds

    Peak specifications are measured under controlled conditions. A controller that reaches its thermal limit during repeated image builds can reduce speed until it cools. Desktop boards often include an M.2 plate; small-form-factor systems may have poor airflow; laptops may not accept a heatsink at all.

    Read the motherboard or laptop service manual before ordering. Confirm M.2 2280 support, PCIe generation, available lanes, single- or double-sided clearance, and whether removing a factory heatsink affects the drive warranty. Do not stack a motherboard plate on top of a factory heatsink.

    On Linux, the open-source nvme-cli tool can read the drive’s SMART and health information. Record temperature and data-unit writes during your normal workload rather than treating one synthetic test as the answer:

    sudo nvme list
    sudo nvme smart-log /dev/nvme0
    

    Replace the device name with the one reported on your machine. These commands inspect the drive; they do not prove that storage is the build bottleneck.

    Measure Cold and Warm Builds Separately

    A useful comparison records at least two cases. A cold build exercises downloads, extraction, compilation, and writes. A warm build shows whether the Dockerfile and cache layout are working. Record the same commit, builder, network conditions, and command each time.

    /usr/bin/time -v docker buildx build --load -t storage-test:latest .
    docker system df
    docker buildx du
    

    Do not clear caches on a production machine merely to create a prettier chart. If a cold-cache test is necessary, use an isolated builder or disposable test host. Also watch CPU utilization and network transfer: an idle SSD during a slow build points elsewhere.

    My Buying Rule for a Docker Workstation

    I would choose capacity first, then endurance, physical compatibility, warranty, and cooling. After those pass, I would compare price. For most PCIe 4.0 development systems, the practical difference between these three drives is more likely to come from capacity headroom and thermal behavior than a small gap in advertised sequential speed.

    If the drive will also hold irreplaceable data, remember that endurance is not a backup. Keep source in version control and maintain a separate backup. My TrueNAS drive guide covers storage roles where redundancy and recovery matter more than workstation build speed, while the homelab hardware guide covers the rest of a self-hosted system.

    Start by measuring cache size and checking your M.2 slot. Then compare the exact capacity and warranty terms instead of buying whichever listing has the biggest MB/s number. For more engineering and market research notes, join Alpha Signal on Telegram.

  • Batch Image Compression in CI with QuickShrink CLI: WebP, AVIF, and Metadata Stripping

    I inspected the QuickShrink CLI package after noticing the same image problem in several web projects: the source tree starts tidy, then screenshots, hero images, and copied assets slowly arrive in mixed formats and oversized dimensions. A browser compressor is useful for one file. A build directory needs a repeatable command.

    QuickShrink CLI turns that job into a local batch step. Version 1.0.0 accepts files, directories, and glob patterns; writes JPEG, PNG, WebP, or AVIF; limits dimensions without enlarging smaller images; and strips metadata unless told otherwise. The package uses Sharp 0.34 and libvips rather than uploading files to a remote compression service.

    Why the Command Line Changes the Workflow

    Manual compression depends on memory. Someone has to remember to open a site, drag in each asset, choose settings, download the result, and place it in the correct directory. That can work for a single blog image, but it is easy to skip when a pull request contains dozens of files.

    A CLI makes the rule executable. The same command can run on a laptop, in an npm script, or in CI. Inputs and output settings live beside the project instead of inside one developer’s browser history. That makes the result easier to repeat when another person checks out the repository.

    This is also a different implementation from browser-side compression. The web version uses browser APIs and is convenient for interactive work. The CLI uses Sharp on top of libvips, so it fits folder processing and build automation. If you want the browser mechanics, my earlier breakdown of Canvas, toBlob, and image compression covers that path.

    Run It Once Without a Global Install

    The hosted package can run through npx, so a global install is optional:

    npx https://quickshrink.orthogonal.info/cli/quickshrink.tgz \
      ./images \
      --out ./dist/images \
      --format webp \
      --quality 80
    

    The input may be a file, a directory, or a quoted glob. Directories are scanned recursively. By default, output goes into ./quickshrink-out, quality is 80, the original format is kept, and the directory structure is preserved. Use --flatten only when duplicate filenames from separate source folders cannot collide.

    The package requires Node.js 18 or newer. A global install is also supported, after which the command is simply quickshrink. I prefer the explicit npx URL in a CI file because it shows exactly where the package comes from. For a long-lived project, pinning and reviewing the downloaded package before use is the safer choice.

    Resize Without Accidentally Enlarging Images

    Width and height limits are often more valuable than another few quality points. A 4000-pixel screenshot does not belong in a 900-pixel content column. QuickShrink exposes both limits:

    quickshrink ./photos \
      --out ./public/photos \
      --format avif \
      --quality 72 \
      --max-width 1600 \
      --max-height 1200
    

    The code reads each image’s dimensions first and adds a resize only when the source exceeds a requested boundary. Sharp receives fit: "inside" and withoutEnlargement: true, so the aspect ratio stays intact and a small input is not scaled upward.

    I ran a smoke test against a synthetic 1800 by 1200 PNG and requested WebP with an 800-pixel width cap. The output metadata reported 800 by 533, which confirms the proportional resize path. The file-size reduction was unusually large because the test image was a single flat color, so that number would be misleading as a photo benchmark. Real savings depend on image detail, source format, and quality.

    Know What Each Encoder Setting Means

    The --quality value is shared across formats, but the encoders do not interpret it identically. For JPEG, the package enables mozjpeg and passes the requested quality. PNG uses compression level 9 with palette mode. WebP and AVIF receive their own quality setting. Treat 80 as a starting point, not proof that every output will look equivalent.

    For a web project, I would begin with WebP at 75 to 82, inspect text edges and gradients, then consider AVIF for large photographic assets. PNG remains useful when exact pixel values or lossless output matter. A dry run helps verify file matching before any bytes are written:

    quickshrink "src/**/*.{jpg,jpeg,png}" \
      --out ./public/assets \
      --format webp \
      --quality 78 \
      --dry-run
    

    The dry-run output lists each planned source and destination. It is especially useful with broad globs, where a typo can select more files than expected.

    Metadata Is Stripped Unless You Keep It

    Sharp removes EXIF and related metadata unless withMetadata() is called. QuickShrink follows that default. Add --keep-metadata when camera details, color profiles, or other embedded fields are required.

    For public screenshots and web photos, removing metadata usually saves space and reduces accidental disclosure. For archival photography, evidence, or a color-managed print flow, discarding it may be the wrong choice. My byte-level EXIF GPS teardown shows why metadata deserves an explicit decision rather than a default nobody notices.

    Put the Command Behind an npm Script

    A named npm script gives local work and CI the same entry point:

    {
      "scripts": {
        "images:build": "quickshrink 'src/images/**/*.{jpg,jpeg,png}' -o public/images -f webp -q 78 -w 1600",
        "images:check": "quickshrink 'src/images/**/*.{jpg,jpeg,png}' -o public/images -f webp -q 78 -w 1600 --dry-run"
      }
    }
    

    Run npm run images:check while adjusting a glob, then use npm run images:build in the build job. The CLI defaults concurrency to the number of CPU cores, and --concurrency can lower that value on a small runner. It processes failures per file, prints a final count, and sets a nonzero exit code when any file fails, which gives CI a useful signal.

    One operational detail matters: do not write output back over the source directory. A separate output tree makes reviews clear and prevents repeated lossy encoding on later builds.

    A Small Local Image Toolkit

    Software is only one part of an image workflow. These are three practical categories I check when the surrounding hardware becomes the bottleneck:

    Full disclosure: those are affiliate links. I may earn a commission from a qualifying purchase at no extra cost to you.

    Use the Browser for One-Offs, the CLI for Repetition

    The web app is the faster choice when one image needs a quick size reduction and visual preview. The CLI is the better fit when the input is a directory, the settings belong in version control, or the same transformation must run on every build.

    Start with the documented examples on the QuickShrink CLI page, run --dry-run, and inspect a few outputs before adding the command to CI. For more privacy-first developer tools and practical engineering notes, join Alpha Signal on Telegram.

  • What SHA-256 Checksums Prove — Verify Files with HashForge

    Last week I downloaded a command-line release and found the familiar pair of links: a binary and a SHA-256 checksum. I dropped the file into HashForge, got a matching digest, and felt reassured for about five seconds. Then I noticed that the binary and its checksum came from the same server.

    If that server had been compromised, an attacker could have replaced both. The matching hash would still be mathematically correct. It just would not answer the question I actually cared about: did this file come from the real publisher?

    That distinction—integrity versus authenticity—is easy to miss. HashForge makes the integrity check quick and private, but the result is only as trustworthy as the reference hash you compare it with.

    What a matching hash actually proves

    A cryptographic hash turns any number of bytes into a fixed-length fingerprint. SHA-256 always returns 256 bits, usually displayed as 64 hexadecimal characters. Change one bit in the input and the output should change unpredictably.

    When HashForge produces the same SHA-256 value that a publisher lists, you have strong evidence that your local file is byte-for-byte identical to the file the publisher hashed. That catches an incomplete download, disk corruption, a broken mirror, or an accidental replacement.

    HashForge performs that work in your browser. Its live code uses the browser’s crypto.subtle.digest() API for SHA-family hashes, and the file stays on your machine. The Web Crypto digest API exposes SHA-1, SHA-256, SHA-384, and SHA-512. It does not expose MD5, so HashForge includes a local JavaScript MD5 implementation for compatibility checks.

    Integrity is not authenticity

    Imagine a release page serves tool.tar.gz and SHA256SUMS from the same origin. An attacker who controls that origin can upload a backdoored archive, calculate its SHA-256 value, and replace the checksum file. Your comparison passes because the two malicious artifacts agree.

    HTTPS helps protect the connection between your browser and the server. It does not prove that the server, build pipeline, maintainer account, or release artifact was trustworthy before the connection began.

    A checksum becomes more useful when the reference arrives through an independent authenticated path: a signed release manifest, a package manager with signed metadata, a maintainer’s verified channel, or a second domain controlled separately. The independence matters more than the visual length of the hash.

    MD5 and SHA-1 need careful language

    MD5 and SHA-1 are broken for collision resistance. In 2017, Google’s and CWI Amsterdam’s SHAttered demonstration produced two visibly different PDF files with the same SHA-1 digest. MD5 collision attacks had become practical much earlier.

    A collision means an attacker can create two different inputs with the same hash. It does not automatically mean they can take any existing file you choose and manufacture a malicious replacement with the same digest; that is a different problem called a second-preimage attack. Chosen-prefix collision techniques are still enough to make MD5 and SHA-1 unacceptable when an adversary can shape both artifacts.

    I keep MD5 in a tool only for identifying old files or matching a legacy checksum, never as proof against an attacker. SHA-256 remains the sensible default for file integrity. No practical SHA-256 collision is known.

    The download check I use with HashForge

    1. Download the artifact from the publisher’s HTTPS release page.
    2. Find the publisher’s SHA-256 value. Prefer a signed checksum manifest or an independent official channel.
    3. Open HashForge and drop in the file. The browser computes the digest locally; the file is not uploaded.
    4. Paste the expected value into the comparison field. Compare the full digest, not the first or last few characters.
    5. If the project offers a signature, verify it as a separate step. A matching checksum does not replace signature verification.

    The local processing matters when the artifact is proprietary, contains customer data, or comes from an internal build. Uploading a confidential binary to a random checksum site creates a new disclosure risk just to answer a local math question.

    Terminal equivalents for repeatable builds

    HashForge is convenient for an occasional manual check. In a build script, I use the operating system’s command so the expected value can be pinned and the job can fail closed:

    # macOS
    shasum -a 256 tool.tar.gz
    
    # Linux
    sha256sum tool.tar.gz
    
    # PowerShell
    Get-FileHash .\tool.zip -Algorithm SHA256

    Do not paste a shortened digest into CI. A full SHA-256 value is only 64 characters, and truncating it deliberately throws away collision resistance. Store the expected value in reviewed source control or consume a signed manifest.

    When a signature is the real answer

    A digital signature binds a digest to a private key. Verification checks both the file and proof that the signer controlled that key. NIST’s Digital Signature Standard guidance describes that authenticity property; modern software projects may use GPG, platform code signing, or Sigstore.

    The remaining question is how you trust the public key or identity. A signature from an unknown key is not useful. Look for a fingerprint on the project’s established site, a verified maintainer identity, a package ecosystem’s trust root, or Sigstore’s identity and transparency-log checks.

    HMAC is different again. It authenticates data between parties that share a secret, which is why I use HashForge’s HMAC panel to debug webhook signatures. A public download page cannot safely give every visitor the HMAC secret, so HMAC is not a substitute for public-key release signing.

    A five-second threat model

    Risk Does a plain SHA-256 comparison help? Better control
    Accidental corruption Yes Checksum comparison
    Broken or stale mirror Yes, with an independent reference Official checksum
    Network tampering Somewhat HTTPS plus an independent checksum
    Compromised download server No, if it hosts both files Signed release manifest
    Malicious publisher or stolen signing key No Reproducible builds, transparency logs, key revocation

    What I trust in practice

    For a low-risk utility, I want HTTPS and a SHA-256 value from the official release page. For an installer, firmware image, wallet, security tool, or production dependency, I look for a signature or signed package metadata as well. For high-impact infrastructure, I also want reproducible builds or a transparency log.

    Use HashForge when you need a fast browser-only checksum, and keep the claim precise: a match proves that two byte sequences agree. It does not tell you who created those bytes.

    Affiliate disclosure: If you want a deeper treatment of hashes, signatures, and real-world cryptographic failures, this Amazon search for Serious Cryptography uses our affiliate tag. We may earn a commission at no extra cost to you.

    For more browser-only developer tools, see the Orthogonal tools page and my field notes on decoding JWTs locally.

    Alpha Signal: Join the free Telegram channel for market intelligence.

  • How JSON Forge Turns “position 4127” Into a Real Error, In Your Browser

    Last month I pasted a 4MB webhook payload into a random online JSON formatter to find why one field was null. The JSON had a customer’s full name, email, a Stripe customer id, and an internal auth token in the headers block. I hit format, got my answer, closed the tab. Then it hit me: I had no idea where that request just went.

    That’s the whole reason I keep coming back to JSON Forge — it runs entirely in the browser, so the payload never leaves the tab. But the privacy angle is only half of it. The part I actually want to talk about is how a browser-only formatter pulls off things people assume need a server: 10MB files without freezing, exact error line/column, and a clickable tree that shows the JSONPath of any node.

    The error message trick every JSON tool should steal

    Native JSON.parse() gives you a miserable error. In V8 you get something like Unexpected token } in JSON at position 4127. Position 4127 of what? Nobody counts characters. You want a line and column.

    The fix is three lines. JSON.parse hands you the absolute character offset in the message, and you turn that into line/col by slicing the input and counting newlines:

    const match = e.message.match(/position (\d+)/);
    if (match) {
      const pos = parseInt(match[1]);
      const before = input.substring(0, pos);
      const line = before.split('\n').length;
      const col  = pos - before.lastIndexOf('\n');
      msg += ` (line ${line}, col ${col})`;
    }

    That’s it. lastIndexOf('\n') finds the start of the current line, subtract from the position, and you have the column. No parser library, no AST. This is exactly what JSON Forge does to turn position 4127 into line 118, col 23. One caveat: the message format isn’t standardized. Firefox says line 118 column 23 directly, Safari phrases it differently. If you ship this, regex the position defensively and fall back to the raw message.

    Why “large file support” is mostly about not painting

    The naive formatter does el.innerHTML = syntaxHighlight(json) and dies on anything past a couple hundred KB — you’re asking the browser to build a DOM node per token. For a 10MB file that’s millions of spans.

    The trick isn’t a Web Worker. Parsing 10MB with JSON.parse takes tens of milliseconds; that’s not your bottleneck. Rendering is. JSON Forge guards the expensive path with a hard cutoff:

    function renderHighlighted(json) {
      // Skip per-token highlighting past 500KB
      if (json.length > 500000) {
        outputEl.textContent = json;   // plain text, one node
        return;
      }
      // ...build highlighted spans for smaller inputs
    }

    Above the threshold you drop to a single text node. You lose color, you keep your framerate. It’s an honest tradeoff and it’s the right one — nobody is visually scanning 10MB of colorized JSON anyway. For structured browsing at that size, the collapsible tree view is what you actually want, because it only builds DOM for nodes you’ve expanded.

    Clicking a node to get its JSONPath

    The tree view builds each node recursively and stamps the path onto the element as it goes. When you click, it reads that path back out — no reverse lookup, no walking parent pointers:

    function buildTreeNode(value, path, depth) {
      // path is passed down: '$', '$.user', '$.user.emails[0]' ...
      el.dataset.path = path;
      el.addEventListener('click', (e) => {
        e.stopPropagation();
        showPath(path);   // renders $ › user › emails › [0]
      });
    }

    Because the path is computed on the way down the recursion, every node already knows its own address. Clicking a deeply nested value gives you $.data.items[3].metadata.tags[0] instantly, which you can paste straight into a jq filter or a JSONPath query in your code. That round trip — visual tree to copyable path — is the thing I use daily.

    The auto-fix is regex, and that’s fine (if you know the edges)

    The “repair broken JSON” button isn’t a grammar-aware parser. It’s a short stack of regex replacements applied in order:

    fixed = fixed.replace(/,(\s*[}\]])/g, '$1');        // trailing commas
    fixed = fixed.replace(/'/g, '"');                    // single → double quotes
    fixed = fixed.replace(/(\{|,)\s*([a-zA-Z_$][\w$]*)\s*:/g, '$1"$2":'); // unquoted keys

    This nails the three things you actually hit copy-pasting from JavaScript source or a Python dict: trailing commas, single quotes, and bare keys. It deliberately does not try to quote unquoted string values — that’s ambiguous with numbers and booleans, so it bails rather than corrupt your data. The single-quote replace is the sharp edge: if a string value legitimately contains an apostrophe ("can't" written as 'can\'t'), a blunt global replace breaks it. So the tool re-runs JSON.parse after fixing and tells you if the repair didn’t actually produce valid JSON, instead of silently handing back garbage. Know the limits and it saves you 30 seconds every time.

    Why browser-only is the real feature

    Here’s the uncomfortable part. Most “free online JSON formatter” sites POST your input to a backend, and a good number log it. I’ve written before about how your online SQL formatter might be logging your database password — the same logic applies double to JSON, because JSON is where the tokens, PII, and API responses live.

    You can verify JSON Forge yourself: open DevTools, go to the Network tab, paste a giant payload, format it. Zero requests. Everything — parse, highlight, tree, JSONPath, auto-fix — happens in JSON.parse and DOM APIs that shipped in your browser years ago. Turn off your Wi-Fi and it still works, because it’s a PWA you can install and run offline. That’s not a marketing checkbox; it’s the difference between a tool you can paste production data into and one you can’t.

    If you want the same guarantee for other formats, the rest of our browser-only set follows the same rule: Base64Lab for encode/decode/preview, DiffLab for side-by-side text compare, and HashForge for hashing. None of them phone home.

    Related reading: Base64Lab for decoding JWTs offline and HashForge for verifying checksums both run entirely client-side, same as JSON Forge.

    Worth having on the shelf

    If you spend your days elbow-deep in JSON payloads and event streams, the book that made me think clearly about the shape of that data is Martin Kleppmann’s Designing Data-Intensive Applications (affiliate link — full disclosure). It’s not about formatting; it’s about why your data looks the way it does across queues, logs, and APIs, and it’s the one systems book I re-read.

    Try JSON Forge the next time you’re about to paste a payload into some random site. Then check the Network tab and see the difference for yourself.


    Join https://t.me/alphasignal822 for free market intelligence.

  • The Frankfurter API: Pull ECB Exchange Rates as JSON (No Key, No Rate Limits)

    I needed to backfill three years of EUR/USD daily closes for a P&L report last month. My first instinct was the usual: sign up for some FX data vendor, wait for the API key email, paste it into a .env file, then discover the free tier caps me at 100 requests a month and doesn’t include historical data anyway. I’ve done this dance a dozen times. This time I didn’t.

    The Frankfurter API gives you European Central Bank reference rates as clean JSON. No key, no signup, no rate limit that I’ve ever hit. It pulls from the ECB’s daily reference rates, which are the same numbers your bank and half the finance industry quote against. Here’s how it works and where it bites.

    The one request that does 90% of the job

    Latest rates, base USD, a couple of currencies:

    curl "https://api.frankfurter.dev/v1/latest?base=USD&symbols=EUR,GBP,JPY"

    You get back exactly what you’d hope for:

    {
      "amount": 1.0,
      "base": "USD",
      "date": "2026-07-22",
      "rates": { "EUR": 0.87658, "GBP": 0.74807, "JPY": 163.07 }
    }

    Drop the symbols param and you get every currency the ECB tracks (about 30). Drop base and it defaults to EUR, since that’s the ECB’s native quote currency. The amount field lets you convert a specific sum in one shot — ?amount=250&base=USD&symbols=EUR tells you what 250 dollars is in euros without you doing the multiply.

    Historical rates on a single date

    Swap latest for an ISO date and you get that day’s fixing:

    curl "https://api.frankfurter.dev/v1/2020-01-01?base=USD&symbols=EUR"
    # {"amount":1.0,"base":"USD","date":"2019-12-31","rates":{"EUR":0.89015}}

    Notice the returned date is 2019-12-31, not the Jan 1 I asked for. That’s the first real gotcha: the ECB doesn’t publish on weekends or TARGET holidays, so Frankfurter rolls back to the last available business day. New Year’s Day has no fixing, so you get December 31st. This is correct behavior, but if you’re joining this against your own date series, align on the returned date, not the one you requested, or you’ll double-count or drop rows.

    Time series — the part I actually came for

    This is where the paid vendors usually put up a paywall. Frankfurter just hands it over with a date range using ..:

    curl "https://api.frankfurter.dev/v1/2024-01-01..2024-01-05?base=USD&symbols=EUR"
    {
      "amount": 1.0,
      "base": "USD",
      "start_date": "2023-12-29",
      "end_date": "2024-01-05",
      "rates": {
        "2023-12-29": { "EUR": 0.90498 },
        "2024-01-02": { "EUR": 0.91274 },
        "2024-01-03": { "EUR": 0.91583 },
        "2024-01-04": { "EUR": 0.91299 },
        "2024-01-05": { "EUR": 0.91567 }
      }
    }

    Weekends are simply absent — there’s no Dec 30/31 or Jan 1 in that response because there was no ECB fixing. You can leave the end date open (2024-01-01..) to pull everything up to today. For a full multi-year backfill that’s one HTTP call, and the payload is small because it’s just floats keyed by date.

    Here’s the Python I used to turn that into a pandas frame:

    import requests, pandas as pd
    
    url = "https://api.frankfurter.dev/v1/2021-01-01..?base=USD&symbols=EUR,GBP,JPY"
    data = requests.get(url).json()["rates"]
    
    df = pd.DataFrame(data).T                # dates become the index
    df.index = pd.to_datetime(df.index)
    df = df.sort_index()
    print(df.tail())

    The .T transpose matters because the JSON is keyed date → currency → rate, and DataFrame reads the outer keys as columns by default. Three lines and I had a clean daily series I could reindex to a business-day calendar and forward-fill for the missing holidays.

    Where it doesn’t fit

    Frankfurter is ECB reference data, so know what that means before you wire it into anything:

    • One price per day, not intraday. The ECB publishes a single reference rate around 16:00 CET. If you need tick data or an FX rate at 09:31:04, this is the wrong tool. It’s for reporting, accounting, and backtests on daily bars — not for trading execution.
    • It’s a reference, not a tradeable quote. There’s no bid/ask spread here. Don’t use it to mark a live position you’d actually close at market.
    • Currency coverage is ECB’s list. Majors and most liquid crosses are there. Exotic or pegged currencies the ECB doesn’t track won’t appear. No crypto either.
    • Rates update once a day, on business days. If your cron hits it at 08:00 CET you’re getting yesterday’s fixing until the new one posts.

    For the report I was building — convert historical foreign revenue to USD at each day’s official rate — those constraints are exactly what I wanted. Auditors like ECB reference rates precisely because there’s one unambiguous number per day.

    Why I trust a keyless API here

    I’m usually suspicious of “no key required” services because the business model is often “we’ll add a paywall once you depend on us.” Frankfurter is open source and the data underneath is the ECB’s public feed, which isn’t going anywhere. If the hosted instance ever disappears, you can self-host it against the same ECB XML and change one hostname. That’s the kind of dependency I’m comfortable building on — the same reason I lean on other keyless government endpoints like SEC EDGAR and the Treasury FiscalData API instead of commercial data brokers.

    If you’re doing this kind of number-crunching regularly, a second monitor pays for itself the first time you’re diffing a rate series against a spreadsheet. I run a cheap Dell 27-inch IPS monitor as a dedicated data pane (full disclosure: affiliate link). Overkill for one API, worth it once you’ve got three terminals of JSON open.

    The whole thing in one script

    import requests
    
    def convert(amount, frm, to, date="latest"):
        url = f"https://api.frankfurter.dev/v1/{date}?base={frm}&symbols={to}"
        r = requests.get(url).json()
        rate = r["rates"][to]
        return round(amount * rate, 2), r["date"]
    
    usd, on = convert(1000, "USD", "EUR", "2023-06-15")
    print(f"$1000 = €{usd} at the ECB fix on {on}")

    No key to rotate, no dashboard to log into, no free-tier counter ticking down. For daily FX in reports and backtests, this is the first thing I reach for now.


    Join https://t.me/alphasignal822 for free market intelligence.

  • Verifying Webhook Signatures by Hand: HMAC-SHA256 in the Browser with HashForge

    A webhook fired at 2am, my handler 500’d, and the vendor’s dashboard just said “delivery failed.” No body, no signature, no clue. When I finally caught the payload, the first thing I needed to know was: is this actually from them, or is someone POSTing garbage at my endpoint? That question is answered by one line of crypto — an HMAC-SHA256 signature — and you can check it by hand in HashForge without pasting a production secret into some random website.

    This post is about the boring, load-bearing part of webhooks that nobody documents well: how the signature header is computed, why your comparison keeps failing, and how to verify one manually when a delivery breaks.

    What the signature header actually is

    Every serious webhook provider signs the request body. GitHub sends X-Hub-Signature-256. Stripe sends Stripe-Signature. Shopify sends X-Shopify-Hmac-Sha256. Different header names, same idea:

    signature = HMAC-SHA256(secret, raw_request_body)

    The provider and you both know a shared secret. They hash the exact bytes of the body with that secret and ship the result in a header. You recompute the same hash on your side. If the two match, the message is authentic and untampered. If they don’t, you reject it with a 401 and move on.

    The reason this matters: your webhook URL is public the moment you register it. Anyone who finds it can POST a fake “payment succeeded” event. Without signature verification, your app will happily believe them.

    Verifying a GitHub signature by hand

    GitHub’s own docs use a concrete example, which makes it perfect for a sanity check. Secret is It's a Secret to Everybody, body is Hello, World!. The expected signature is:

    757107ea0eb2509fc211221cce984b8a37570b6d7586c22c46f4379c8b043e17

    Open HashForge, switch to the HMAC panel, drop the secret into the key field and the body into the message field, pick SHA-256. You get exactly that hex string. That’s the whole verification. GitHub prefixes it with sha256= in the header, so the real value on the wire is sha256=757107ea... — strip the prefix before comparing.

    I like doing this in HashForge specifically because it runs entirely in the browser. The HMAC is computed with the Web Crypto API (crypto.subtle.sign), so your secret never leaves the tab. Check the Network panel if you don’t believe me — there are no outbound requests. Pasting a webhook secret into a server-side “online HMAC generator” is the kind of thing that ends up in someone’s access logs.

    The three reasons your comparison fails

    Manual verification exposes the bugs that silently break webhook handlers. In order of how often they’ve bitten me:

    1. You hashed the parsed body, not the raw body. This is the big one. Frameworks love to parse JSON for you. But JSON.stringify(JSON.parse(body)) is not the original bytes — key order changes, whitespace vanishes, unicode gets re-escaped. The signature is over the exact bytes the provider sent. In Express you need the raw buffer:

    app.post('/webhook',
      express.raw({ type: 'application/json' }),
      (req, res) => {
        const sig = req.get('X-Hub-Signature-256');
        const expected = 'sha256=' + hmacSha256(secret, req.body);
        // req.body is a Buffer here, not a parsed object
      }
    );

    If your handler works in tests but fails in production, this is almost always why — a body parser upstream mangled the bytes before you hashed them.

    2. Wrong key encoding. Most providers treat the secret as a UTF-8 string. Some — a few payment and banking APIs — give you a hex or base64 secret that must be decoded to raw bytes first. Hashing the literal hex characters instead of the decoded bytes gives a completely different result. If HashForge’s output doesn’t match and you’re sure the body is right, this is the next thing to check.

    3. Non-constant-time comparison. Once the bytes are right, don’t compare signatures with ===. A naive string compare returns early on the first mismatched character, which leaks timing information an attacker can measure. Use a constant-time compare:

    const crypto = require('crypto');
    function safeEqual(a, b) {
      const ba = Buffer.from(a), bb = Buffer.from(b);
      if (ba.length !== bb.length) return false;
      return crypto.timingSafeEqual(ba, bb);
    }

    Stripe adds a timestamp — and so should you

    Stripe’s Stripe-Signature header isn’t just the HMAC. It looks like:

    t=1699999999,v1=5257a869e7ecebeda32affa62cdca3fa...

    The signed payload is t + "." + body, not the body alone. So you concatenate the timestamp, a literal dot, and the raw body, then HMAC-SHA256 that whole string with your signing secret. To reproduce it in HashForge, paste 1699999999.{your raw body} into the message field.

    The timestamp exists to stop replay attacks. Someone who captures a valid signed request can’t resend it a day later, because you also check that t is within a few minutes of now. If you’re building your own webhook sender, copy this pattern — sign the timestamp alongside the body and reject stale ones.

    When to reach for manual verification

    You don’t do this on every request — your code handles the happy path. Manual HMAC checking earns its keep in exactly three moments:

    • First integration. Before you trust your verification code, confirm it against a known payload. Recompute the signature in HashForge and diff it against what your handler produced. If they disagree, your handler is wrong, not the provider.
    • A specific delivery failed. Grab the raw body and the signature header from the provider’s delivery log, recompute by hand, and you’ll immediately see whether it’s a body-encoding bug or a genuinely bad signature.
    • Rotating secrets. After changing a signing secret, verify one real event manually before you trust the pipeline again.

    If you want the byte-level view of what’s actually being hashed, the Web Crypto API is worth understanding — it’s the same primitive HashForge uses under the hood. And if you’re inspecting the JWTs some webhooks carry instead of HMAC headers, the offline JWT reader covers that failure mode.

    Keep the secret on paper, not in a note app

    One habit worth building: webhook signing secrets are long-lived credentials, and they end up scattered across .env files, CI variables, and password managers. I keep the master copy of anything I can’t regenerate in a small hardware-backed spot rather than a cloud note. A cheap encrypted USB key for offline secret backups has saved me twice when a password manager sync went sideways. Full disclosure: that’s an Amazon affiliate link.

    The point of verifying signatures at all is that you don’t get to be sloppy with the secret. Compute the HMAC in the browser, compare in constant time, hash the raw bytes, and check the timestamp. Four rules, and your webhook endpoint stops trusting strangers.

    You can try the HMAC verification yourself right now in HashForge — pick SHA-256, paste a secret and a message, and watch the signature appear without a single network request.


    Join https://t.me/alphasignal822 for free market intelligence.

    Related reading

  • Your Password Generator Is Only as Good as crypto.getRandomValues

    A few weeks ago I watched a teammate generate a “secure” password with a little snippet he’d written years ago. It looked fine — 16 characters, mixed case, symbols. Then I asked him what was seeding it. He shrugged: Math.random(). That password had far less real randomness than it looked like, and neither of us could tell by staring at it. That’s the whole problem with password strength — the weakness is invisible.

    So I want to walk through what actually makes a generated password strong, using PassForge, a browser-only password generator I use. It runs entirely client-side, and because the whole thing is a single HTML file with the logic in plain view, it’s a good way to show the math instead of hand-waving about it.

    Why Math.random() is quietly broken for passwords

    JavaScript’s Math.random() is a pseudo-random number generator. It’s fast, it’s fine for shuffling a card animation, and it is not cryptographically secure. The output is deterministic given the internal state, and in V8 that state is only 128 bits seeded in a way that was never meant to resist an attacker. If someone can observe a few outputs, they can predict the rest.

    The fix is the Web Crypto API. Here’s exactly how PassForge pulls a random integer — no library, just the browser’s CSPRNG:

    function cryptoRandInt(max) {
      const arr = new Uint32Array(1);
      crypto.getRandomValues(arr);
      return arr[0] % max;
    }

    crypto.getRandomValues is backed by the operating system’s secure random source. That’s the same class of entropy your TLS handshakes use. The % max introduces a tiny modulo bias when max doesn’t evenly divide 2³², but for a 26- or 33-character pool that bias is negligible — we’re talking a fraction of a fraction of a bit. For a password generator, this is the correct baseline, and it’s the first thing I check before trusting any generator.

    Entropy is the only number that matters

    “Strong password” is a marketing phrase. Entropy in bits is the actual measurement. It answers one question: how many guesses, on average, before an attacker lands on your password?

    For a random character password, entropy is length × log2(poolSize). PassForge computes the pool the honest way — by inspecting which character classes are actually present:

    function calcEntropy(password) {
      let poolSize = 0;
      if (/[a-z]/.test(password)) poolSize += 26;
      if (/[A-Z]/.test(password)) poolSize += 26;
      if (/[0-9]/.test(password)) poolSize += 10;
      if (/[^a-zA-Z0-9]/.test(password)) poolSize += 33;
      return Math.floor(password.length * Math.log2(poolSize));
    }

    So a 16-character password using all four classes has a pool of 95, and log2(95) ≈ 6.57 bits per character — about 105 bits total. Drop the symbols and you’re at a pool of 62, roughly 95 bits. That 10-bit gap is a factor of 1,024 in guessing difficulty, from one design choice most people never think about.

    Passphrases: fewer characters, more entropy per word

    Here’s the part that surprises people. A four-word passphrase like correct-horse-battery-staple feels weaker than Kx9$mQ2!, but it usually isn’t. PassForge uses the EFF short wordlist — 1,296 words — and each randomly chosen word contributes log2(1296) ≈ 10.34 bits:

    function passphraseEntropy(wordCount, addNumber, addSymbol) {
      let bits = wordCount * Math.log2(WORDS.length);
      if (addNumber) bits += Math.log2(100);
      if (addSymbol) bits += Math.log2(8);
      return Math.floor(bits);
    }

    Six words gives you about 62 bits before any decoration — and it’s a string you can actually retype from memory when a password manager isn’t handy. The key word is randomly. If you pick the words yourself, the entropy math collapses, because human word choice is predictable. The security comes entirely from crypto.getRandomValues picking the index for you.

    The crack-time number, and its honest assumptions

    PassForge turns entropy into a human-readable estimate, and I like that it states its threat model in the code instead of hiding it:

    function crackTimeStr(bits) {
      // Assume 10 billion guesses/sec (modern GPU cluster)
      const guesses = Math.pow(2, bits);
      const seconds = guesses / 1e10;
      ...
    }

    Ten billion guesses per second is a reasonable stand-in for an offline attack against a fast hash. That assumption matters. Against a properly slow hash like bcrypt or Argon2, the attacker’s rate drops by orders of magnitude, so the tool is being conservative — it’s modeling the worst realistic case, not the best. I trust a calculator that’s pessimistic on my behalf more than one that shows “3 trillion years” for a mediocre password.

    At 10 billion guesses/sec, a 60-bit password falls in a few years, an 80-bit password holds for millions of years, and once you clear 128 bits you’re past “longer than the age of the universe.” That’s why the target isn’t a length — it’s a bit count.

    The small touches that show someone thought about it

    Two details I appreciated. First, the ambiguous-character filter: O0lI1| get stripped when you ask for it, so you never squint at a password wondering if that’s a one or a lowercase L. Second, the generator guarantees at least one character from each selected class and then shuffles the result with a crypto-backed Fisher-Yates, so the guaranteed characters don’t always land in front. Skipping that shuffle is a classic subtle bug that leaks a little entropy and makes output patterns predictable.

    Why browser-only is the right call here

    A password generator that sends anything over the network is a contradiction. PassForge is one static HTML file — you can open the page, hit “save,” disconnect from Wi-Fi, and it keeps working. Nothing to install, nothing phoning home, and you can read the entire source before you trust it. That’s the same reason I use its siblings for other one-off tasks: HashForge for hashing and Base64Lab for encoding, all client-side. I wrote separately about why I stopped uploading files to free online tools — the logic applies double to secrets.

    If you’d rather not trust any web page with generation at all, a hardware option is the cleanest boundary: I keep a YubiKey 5 NFC for the accounts that matter, and for storing what I generate a manager like 1Password is worth the price. Full disclosure: those are affiliate links.

    The takeaway is simpler than the math looks: check that your generator uses crypto.getRandomValues, aim for 80+ bits of entropy, and let the tool pick the randomness — never your own brain. Try PassForge here and watch the entropy number move as you change the settings.


    Join https://t.me/alphasignal822 for free market intelligence.

    Related reading

Also by us: StartCaaS — AI Company OS · Hype2You — AI Tech Trends