Docker DBA Lab

https://percona.community/blog/2026/10/dba_docker_lab_hu_d6c3281482a4c3e5.jpg

Building a Database DBA Lab with Docker, PostgreSQL, MySQL, and Percona Monitoring and Management

How I built a local database playground, learned a few lessons the hard way, and created a practical command reference along the way.

Introduction

One of the best ways to learn database administration is to get your hands dirty.

Documentation is important, but there’s something different about building your own environment, breaking it, troubleshooting it, and eventually understanding why everything works.

Recently, I decided to create a local database lab using Docker Desktop on my Mac. My goal was straightforward: build an environment where I could experiment with MySQL, PostgreSQL, Percona XtraDB Cluster (PXC), and Percona Monitoring and Management (PMM), without worrying about disrupting a production environment.

What started as a few containers quickly became a much more interesting learning experience.

Along the way, I ran into Docker networking problems, container persistence challenges, cluster bootstrap issues, and PostgreSQL monitoring configuration. This article shows the setup I recommend after working through those problems, so you can start with the right foundation instead of discovering it at the end.

1. Building the Lab

The foundation of my environment is Docker Desktop.

The lab includes:

  • Percona Server for MySQL 8.4
  • A three-node Percona XtraDB Cluster
  • PostgreSQL 17
  • PMM Server and PMM Client
  • Persistent Docker volumes
  • A shared Docker network

The architecture looks like this:

text

Docker Desktop
│
├── pmm-network
│ ├── pmm-server
│ ├── pmm-client
│ ├── percona-server
│ ├── postgres-server
│ ├── pxc1
│ ├── pxc2
│ └── pxc3
│
└── Persistent Docker Volumes
 ├── pmm-data
 ├── pmm-client-data
 ├── mysql-data
 ├── postgres-data
 ├── pxc1_data
 ├── pxc2_data
 └── pxc3_data

There are two ways to build this.

Option A (recommended): Docker Compose

Docker Compose lets you define every container, volume, and network attachment in one file. The whole lab becomes reproducible: one command brings it up, one command shuts it down, and nothing depends on remembering which docker run flags you used weeks ago.

I started out with individual docker run commands and later moved everything to Compose. If I were starting again, I would begin here.

First, keep passwords out of the Compose file by putting them in a .env file next to it (and out of version control):

bash

# .env
MYSQL_ROOT_PASSWORD=change-me
PXC_ROOT_PASSWORD=change-me
POSTGRES_PASSWORD=change-me
PMM_ADMIN_PASSWORD=change-me
PG_MONITOR_PASSWORD=change-me
MYSQL_MONITOR_PASSWORD=change-me

Then define the lab in docker-compose.yml:

yaml

services:
 pmm-server:
 image: percona/pmm-server:3.9.1
 platform: linux/amd64
 container_name: pmm-server
 ports:
 - "443:8443"
 volumes:
 - pmm-data:/srv
 networks: [pmm-network]

 pmm-client:
 image: percona/pmm-client:3.9.1
 container_name: pmm-client
 depends_on: [pmm-server]
 environment:
 PMM_AGENT_SERVER_ADDRESS: pmm-server:8443
 PMM_AGENT_SERVER_USERNAME: admin
 PMM_AGENT_SERVER_PASSWORD: ${PMM_ADMIN_PASSWORD}
 PMM_AGENT_SERVER_INSECURE_TLS: 1
 PMM_AGENT_SETUP: 1
 PMM_AGENT_CONFIG_FILE: config/pmm-agent.yaml
 volumes:
 - pmm-client-data:/usr/local/percona/pmm/config
 networks: [pmm-network]

 percona-server:
 image: percona/percona-server:8.4.11
 container_name: percona-server
 environment:
 MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
 ports:
 - "3306:3306"
 volumes:
 - mysql-data:/var/lib/mysql
 networks: [pmm-network]

 postgres-server:
 image: postgres:17
 container_name: postgres-server
 environment:
 POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
 command: >
 postgres
 -c shared_preload_libraries=pg_stat_statements
 -c track_io_timing=on
 ports:
 - "5432:5432"
 volumes:
 - postgres-data:/var/lib/postgresql/data
 networks: [pmm-network]

 pxc1:
 image: percona/percona-xtradb-cluster:8.4.11
 container_name: pxc1
 environment:
 CLUSTER_NAME: pxc-lab
 MYSQL_ROOT_PASSWORD: ${PXC_ROOT_PASSWORD}
 volumes:
 - pxc1_data:/var/lib/mysql
 - ./certs:/etc/mysql/certs:ro
 - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro
 stop_grace_period: 120s
 healthcheck:
 test: ["CMD", "mysqladmin", "ping", "-uroot", "-p${PXC_ROOT_PASSWORD}"]
 interval: 10s
 retries: 30
 networks: [pmm-network]
 ports:
 - "3307:3306"

 pxc2:
 image: percona/percona-xtradb-cluster:8.4.11
 container_name: pxc2
 environment:
 CLUSTER_NAME: pxc-lab
 CLUSTER_JOIN: pxc1
 MYSQL_ROOT_PASSWORD: ${PXC_ROOT_PASSWORD}
 volumes:
 - pxc2_data:/var/lib/mysql
 - ./certs:/etc/mysql/certs:ro
 - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro
 stop_grace_period: 120s
 depends_on:
 pxc1:
 condition: service_healthy
 networks: [pmm-network]
 ports:
 - "3308:3306"

 pxc3:
 image: percona/percona-xtradb-cluster:8.4.11
 container_name: pxc3
 environment:
 CLUSTER_NAME: pxc-lab
 CLUSTER_JOIN: pxc1
 MYSQL_ROOT_PASSWORD: ${PXC_ROOT_PASSWORD}
 volumes:
 - pxc3_data:/var/lib/mysql
 - ./certs:/etc/mysql/certs:ro
 - ./conf.d:/etc/percona-xtradb-cluster.conf.d:ro
 stop_grace_period: 120s
 depends_on:
 - pxc2
 networks: [pmm-network]
 ports:
 - "3309:3306"

networks:
 pmm-network:
 name: pmm-network

volumes:
 pmm-data:
 pmm-client-data:
 mysql-data:
 postgres-data:
 pxc1_data:
 pxc2_data:
 pxc3_data:

A few design choices are worth calling out, because each one comes from a problem I hit:

  • Volumes and the network are declared up front. Every database has persistent storage and is attached to pmm-network from the moment it is created, so there is nothing to wire up afterward.
  • All PXC nodes share the same SSL certificates. Mismatched certificates were behind several of my early cluster join failures.
  • stop_grace_period: 120s gives each PXC node time to shut down cleanly instead of being killed (exit code 137).
  • depends_on defines the PXC order. Compose starts pxc1 first and waits for it to be healthy before the others join. On shutdown, Compose stops dependents first, so the nodes leave in the order pxc3, pxc2, pxc1.

Bring the lab up and down with:

bash

docker compose up -d
docker compose stop # stops containers, keeps everything
docker compose down # removes containers, keeps named volumes

Option B (alternative): Individual docker run commands

If you prefer to build containers one at a time, or want to understand what Compose is doing for you, you can create the network yourself and attach containers to it:

bash

docker network create pmm-network

docker run -d --name postgres-server \
 --network pmm-network \
 -e POSTGRES_PASSWORD="$POSTGRES_PASSWORD" \
 -v postgres-data:/var/lib/postgresql/data \
 -p 5432:5432 \
 postgres:17

An existing container can be attached later:

bash

docker network connect pmm-network postgres-server

This works, but you are responsible for recreating each container with exactly the same flags, volumes, and network settings every time. That is the main reason I recommend Option A. Whichever option you choose, avoid mixing the two for the same lab, because it becomes hard to tell which containers are managed by Compose and which are not.

A networking lesson that applies either way

At one point, PMM couldn’t resolve my PostgreSQL container:

text

lookup postgres-server on 127.0.0.11:53: no such host

The problem wasn’t PostgreSQL or PMM. The PostgreSQL container was stopped, so it had no active network endpoint, and Docker’s internal DNS had nothing to return.

That was an important reminder: always verify the container and network state before troubleshooting the application.

2. Making Monitoring Persistent with PMM

Once the databases were running, I wanted visibility into their performance.

PMM provides dashboards and metrics for MySQL, PXC, and PostgreSQL. It consists of a server that stores and visualizes monitoring data and a client agent that collects database metrics.

The key thing to understand is that persistent storage and service registration are related, but they are not the same thing.

PMM Server maintains the monitoring inventory, while PMM Client has its own configuration and agent identity. If the client is recreated with a new identity, the association between the client and its registered services is lost. In my early setup, after recreating containers, the client listing showed only the basic agents:

text

pmm_agent Connected
node_exporter Running
vmagent Running

Two things prevent this, and both should be in place from the start:

  1. A persistent volume for the client configuration. In the Compose file above, pmm-client-data keeps the agent identity across restarts and recreations.
  2. An idempotent registration script. It adds each database only if it isn’t already registered, so it is safe to run after every docker compose up.

Before registering, create a monitoring user in each database. For PostgreSQL, for example:

sql

CREATE USER pmm WITH PASSWORD 'change-me';
GRANT pg_monitor TO pmm;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

Then the registration script, which reads credentials from the .env file rather than hardcoding them:

bash

#!/usr/bin/env bash
# register-pmm.sh
set -euo pipefail
source .env

registered() {
 docker exec pmm-client pmm-admin list | grep -qw "$1"
}

if ! registered postgres-server; then
 docker exec pmm-client pmm-admin add postgresql \
 --username=pmm \
 --password="$PG_MONITOR_PASSWORD" \
 --host=postgres-server \
 --port=5432 \
 postgres-server
fi

for node in percona-server pxc1 pxc2 pxc3; do
 if ! registered "$node"; then
 docker exec pmm-client pmm-admin add mysql \
 --username=pmm \
 --password="$MYSQL_MONITOR_PASSWORD" \
 --host="$node" \
 --port=3306 \
 --query-source=perfschema \
 "$node"
 fi
done

docker exec pmm-client pmm-admin list

The takeaway: a monitoring system should be designed to survive a restart, not just work immediately after installation.

3. Learning PostgreSQL Through Real Workloads

With PostgreSQL running and connected to PMM, I wanted to do more than look at an idle dashboard.

I used sysbench to generate a workload against PostgreSQL 17.

My test environment included ten tables with approximately 750,000 rows each, giving me about 7.5 million rows to experiment with.

A sample workload command looked like this:

bash

sysbench \
 --db-driver=pgsql \
 --pgsql-host=127.0.0.1 \
 --pgsql-port=5432 \
 --pgsql-user=postgres \
 --pgsql-db=sbtest \
 --tables=10 \
 --table-size=750000 \
 --threads=16 \
 --time=300 \
 --report-interval=5 \
 oltp_read_write run

With the workload running, PMM became much more interesting.

I could observe database activity, disk latency, query execution statistics, and changes in resource utilization.

This also helped me understand an important distinction: PostgreSQL tuple activity is not the same thing as SQL queries per second.

One query can read or return many tuples. Looking only at tuple counts without understanding the underlying queries can lead to incorrect conclusions about workload intensity.

4. Understanding PostgreSQL Query Performance

One of the most useful PostgreSQL features for troubleshooting is EXPLAIN.

For example:

sql

EXPLAIN (ANALYZE, BUFFERS)
SELECT *
FROM sbtest1
WHERE id BETWEEN 100 AND 200;

This shows the execution plan, actual execution measurements, and buffer activity.

I started paying attention to several key pieces of information:

  • Whether PostgreSQL used a sequential scan or an index
  • Estimated versus actual rows
  • Execution time
  • Buffer hits and reads
  • Number of execution loops

Because the Compose file loads pg_stat_statements at startup, query statistics are aggregated from the beginning.

A useful query is:

sql

SELECT
 calls,
 total_exec_time,
 mean_exec_time,
 rows,
 LEFT(query, 100) AS query
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;

This provides a different perspective from looking at individual slow queries.

A query that takes only a fraction of a millisecond might still consume substantial cumulative execution time if it runs hundreds of thousands of times.

For a DBA, both individual latency and aggregate workload matter.

5. MVCC, Dead Tuples, and VACUUM

My next learning objective was understanding PostgreSQL’s Multi-Version Concurrency Control (MVCC).

Unlike an in-place overwrite model, PostgreSQL commonly creates a new row version when a row is updated.

Older versions may remain until PostgreSQL determines they are no longer needed.

That leads directly to one of PostgreSQL’s most important maintenance mechanisms: VACUUM.

I experimented with updating large numbers of rows:

sql

UPDATE sbtest1
SET k = k + 1
WHERE id <= 300000;

Then checked the table statistics:

sql

SELECT
 relname,
 n_live_tup,
 n_dead_tup,
 last_autovacuum,
 autovacuum_count
FROM pg_stat_user_tables
WHERE relname = 'sbtest1';

This gave me a practical way to observe dead tuple estimates and autovacuum activity.

I also explored what happens when a long-running transaction holds an old snapshot.

By starting a REPEATABLE READ transaction and leaving it open while another session updated rows, I could demonstrate how PostgreSQL may need to retain older row versions.

That helped connect several important DBA concepts:

Long-running transactions → old snapshots → delayed tuple cleanup → potential table bloat.

I also learned that normal VACUUM generally makes space reusable within PostgreSQL rather than immediately shrinking the table file.

VACUUM FULL can reclaim disk space by rewriting a table, but it requires an exclusive lock and is not something to run casually in production.

6. Operating the PXC Cluster Safely

My three-node Percona XtraDB Cluster taught me that database containers need database-aware operational procedures. That’s why startup and shutdown are built into the Compose file rather than left to chance.

Early on, I ran into certificate problems, donor timeouts, cluster membership issues, and shutdowns that ended with exit code 137 (the container being killed before MySQL finished). The shared certificate directory, the healthcheck on pxc1, and stop_grace_period address those directly.

The shutdown procedure is simply:

bash

docker compose stop pxc3 pxc2 pxc1

If you use Option B, do the same thing explicitly, one node at a time, with a long timeout:

bash

docker stop -t 120 pxc3
docker stop -t 120 pxc2
docker stop -t 120 pxc1

To start the cluster again, check grastate.dat on each node and understand the meaning of safe_to_bootstrap:

bash

docker run --rm -v pxc1_data:/data alpine cat /data/grastate.dat

After a clean shutdown, the last node to leave the cluster (here, pxc1) will have safe_to_bootstrap: 1 and is the right node to start first.

After an unclean shutdown, selecting the correct bootstrap node requires recovery checks. Compare the recovered sequence numbers across nodes and bootstrap from the most advanced one. Simply changing safe_to_bootstrap to 1 without verifying that can lose data.

7. My Most Useful Commands

I kept a command cheat sheet from the start of the project, and it has become one of the most useful outcomes. It captures the commands I actually needed while learning and troubleshooting. I recommend starting one on day one.

For Docker:

bash

# Running containers
docker ps

# All containers
docker ps -a

# Status of the Compose-managed lab
docker compose ps

# Follow logs
docker logs -f postgres-server

# Inspect network configuration
docker inspect postgres-server \
 --format ''

# Resource usage
docker stats

# List Docker volumes
docker volume ls

# Check PMM services
docker exec pmm-client pmm-admin list

These commands are my first line of investigation whenever something isn’t behaving as expected.

For PostgreSQL, my daily reference includes:

sql

-- Current sessions
SELECT pid, usename, state, query
FROM pg_stat_activity;

-- Long-running transactions
SELECT
 pid,
 state,
 now() - xact_start AS transaction_age,
 query
FROM pg_stat_activity
WHERE xact_start IS NOT NULL;

-- Dead tuples
SELECT relname, n_live_tup, n_dead_tup
FROM pg_stat_user_tables
ORDER BY n_dead_tup DESC;

-- Blocking sessions
SELECT
 pid,
 pg_blocking_pids(pid) AS blocked_by
FROM pg_stat_activity
WHERE cardinality(pg_blocking_pids(pid)) > 0;

These are simple queries, but they form a useful starting point for troubleshooting.

8. Design Principles Behind the Lab

To summarize, these are the principles the setup above is built on:

  1. Define the whole lab in Docker Compose, rather than mixing docker run commands with Compose-managed containers.
  2. Declare persistent volumes and network attachments before creating any databases.
  3. Treat startup and shutdown as part of the design, especially for PXC.
  4. Make PMM registration reproducible with a persistent client volume and an idempotent script, and keep credentials out of scripts and command history.
  5. Keep a command cheat sheet from the beginning.

Final Thoughts

What began as an effort to build a small local database environment turned into a practical DBA learning experience.

I now have a place to explore PostgreSQL performance, MVCC, autovacuum, MySQL replication concepts, PXC clustering, and database monitoring without risking a production system.

Perhaps the biggest lesson is that the most valuable learning moments often happen when something breaks.

A Docker DNS failure teaches networking. A missing PMM registration teaches persistence and identity. A PXC bootstrap problem teaches cluster recovery. A growing dead tuple count teaches PostgreSQL MVCC.

And once you understand why something broke, you’re much better prepared to troubleshoot it the next time.

My next goal is to continue expanding the lab with PostgreSQL replication, backup and recovery, and more deliberate performance troubleshooting exercises.


This article describes a local learning environment. The commands and configurations are intended for experimentation and should be reviewed and adapted before use in production. ∎

Planet for the MySQL Community

MySQL Workbench 26.7

https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNOm1nV5-sOyOlra6B3XU3XpTycZ2Wbcxw6SMB48x6pnDd212OTqXwBaNEGXel4tkZzHWZa5YA_zr76jDSxdkoXSOwmiJJmQNYnscfyP7aqjCBkAbjAfZ5wFIU63iXnxuHInOKCAqZ88R_XJqGwNzrLHd44LzMiBw_byEJSKhIaa8-8-i7bEzgQav1SQqF/w1200-h630-p-k-no-nu/Screenshot%20From%202026-10-08%2010-49-02.png

 The old MySQL Workbench was ‘retired’ in January. The latest was recent released without a lot of fanfare. The new version is not an evolution of the old and, in some ways, is lacking features like relationship maps and reverse engineering capabilities. Now server versions 8.0 and earlier may work, but are not supported.

The opening screen of the newest MySQL Workbench

New Look

The new version is built on the top of MySQL Shell. A lot of the features from that product are still around, but the look and feel will not feel familiar to the fans of the shell.

By The Way: The MySQL Shell for VS Code has also been discontinued. 

Installation can be made from any of the usual ways from the usual sources. This is still the high quality package we have come to expect.

SQL Work

What you see when you are working with SQL

Connections are made from the first page by clicking on the plus ‘+’ button and providing the connection details. 

From there click on the box with your connection name. 

Now you can write queries to your heart’s content. You will have to click on the blue ‘Run’ button to get the query to execute. Or, use the Run menu.

The GUI is minimalist, to put it nicely. 

Under the Run menu, you can ‘run’ the query and use Explain, either text or visual. The visual explain was a highlight of the previous version. 

The auto-completion is improved, but other products were much better earlier.   

I wanted to show examples of the visual explain, but I only received ‘No database selected’ messages. Supposedly it supports the hypergraph optimizer and specialized HeatWave Query plans. But not on a local instance with a query that ran well with the non-visual explain options. Frustrating.

Notebooks

Notebooks bring the ability to use JavaScript, TypeScript, or SQL to execute code blocks. This was popular in the MySQL Shell.

I liked the Python capabilities of the previous edition. 

Notebooks provide for JavaScript, TypeScript, and SQL scripting

Having to hit the Run button or use the Run menu to execute queries gets old very quickly. 

Again, minimalist, and very thin compared to the older version. In this age of observation of all things being key, this is disappointing.                

    
Server Status

Performance

Well, the older version had a lot of interesting and sometimes quirky information on system performance. 

Some performance information is available

Prepared to be nostalgic again as they are missing from this new version.  This an no ERM maps remove capabilities where Workbench used to be top notch. 

Conclusion

I have to inform you that I am an employee of DBeaver, which is a competing product. Just as skipping is a competitor to a race car at the Indy 500. 

This version of MySQL Workbench is too stark and minimalist in design for me. It lacks too many features. 

And it is MySQL 8.0 (maybe) or later only. 

I do not know if Oracle/MySQL will add features in the future. This new project is open-source, but will it attract developers to help improve it? Go into ‘not holding my breath mode’ for both cases here.

I will check back when there is a new release, but MySQl Workbench 26.7 is not going into my tool box.

Planet for the MySQL Community

You can turn off & delete Apple Intelligence with an app now

We recently wrote about RemoveMacAI, a tool to disable Apple Intelligence and delete the models, to help save about 13 gigabytes of storage space. While it was initially provided as Terminal commands, you can now download an app that gives you a friendly UI to use.

Be aware that it isn’t a notarized app, so you’ll have a few more hoops to jump through to run it.

Continue Reading on AppleInsider | Discuss on our ForumsAppleInsider News

Sonnet Thunderbolt 5 enclosure gives Mac mini & MacBook Pro PCI-E expansion

https://media.appleinsider.com/gallery/69262-146090-4B6E3B65-E64C-4F50-B420-A37E3F4428C4-xl.jpgSonnet’s Echo SE I T5 gives Mac mini and MacBook Pro users a way to add the kind of PCIe hardware normally reserved for larger workstations without giving up the compact Mac they already have.

Black SonnetTech Echo SE I computer hardware enclosure standing on small feet, with a perforated front panel, against a gradient blue and purple studio backgroundSonnet’s Echo SE I T5

The Thunderbolt 5 enclosure supports video capture cards, professional audio hardware, RAID controllers, and high-speed networking adapters. Sonnet says the system can deliver up to 6,800 MB/s of bidirectional PCIe bandwidth over Thunderbolt 5.

Thunderbolt 5 gives demanding PCI-E cards substantially more bandwidth than older Thunderbolt connections can provide.

The Echo SE I T5 gives compact Macs a practical path to PCIe expansion without making bandwidth the whole story. Because the Mac mini and MacBook Pro don’t have internal PCIe slots, adding specialized hardware usually means using an external enclosure or moving to a larger workstation.

Continue Reading on AppleInsider | Discuss on our ForumsAppleInsider News

F1 Champion vs. Amateur Kart Racers

https://theawesomer.com/photos/2026/10/max_verstappen_vs_go_karts_t.jpg

F1 Champion vs. Amateur Kart Racers

Getting around a race track quickly depends as much on the driver’s skill as the car’s capabilities, if not more so. Red Bull and Max Verstappen prove that point, as the F1 World Champ starts at the back of the pack behind 100 go-kart racers and aims to pass every one of them. The race kicks off at the 9-minute mark, with a thrilling battle to the finish.

The Awesomer

A Super-Easy-to-Install Add-On Clip for Swiss Army Knives

https://s3files.core77.com/blog/images/1852642_81_145566_BnU8uIPv9.jpg

When you think of the classic Swiss Army Knife, you likely envision the common 91mm version. That’s the size that became the standard in the 1950s. As ubiquitous as this line is, the strange thing is that Victorinox never updated them with clips, which became popular with other folding knives starting in the 1980s. So if you’ve got a 91mm SAK, you either fish for it in a pocket, or attach a lanyard to its key ring.

Now Tortoise Gear, a licensed accessory maker for Victorinox, is rectifying the omission.

Their Lamprey91 is an add-on clip that you can install without disassembling the knife.

The clip slides into the thin seam between the handle and internal metal liner, than seats around one of the knife’s concealed rivets. Here inventor and Tortoise Gear founder Eric McCormick demonstrates how to install it:

That’s an impressive piece of remedial design!

The Lamprey91 has been successfully Kickstarted, with 23 days left to pledge at press time. They run $25 and are expected to ship in February. 

Core77

Command-line tool quickly removes Apple Intelligence from macOS 27

https://cdn.arstechnica.net/wp-content/uploads/2026/03/Screenshot-2026-03-03-at-10.30.27-AM-1152×648.jpeg

A new tool is allowing Mac users the ability again to easily turn off Apple Intelligence with one click and free up to 12GB of storage.

Unlike with previous versions of macOS, macOS 27 Golden Gate doesn’t have a toggle for turning off Apple Intelligence. That makes disabling AI features that you may not want more difficult. It also means that the AI models necessary for running Apple Intelligence will take up space on your disk, even if you don’t use them or if you go through your settings to individually find and disable AI capabilities.

In response, a developer known as Om Lahore on GitHub last week created RemoveMacAI, a command-line tool that allows macOS 27 users to “turn off Apple Intelligence on macOS 27" in a way that is “fully reversible,” per the GitHub page. They said that Apple’s Intelligence models take up "about 12GB," but the models can actually take up over 30GB, The Verge noted.

Read full article

Comments

Ars Technica – All content

Movie Finder: semantic search over 1M films, inside MySQL

https://askdba.net/wp-content/uploads/2026/10/myvector_movies-Large.jpeg

Type “a heist that goes wrong in a snowy town” and get back real films, with posters, in a few milliseconds. The vector search behind it runs inside MySQL, with no separate vector database.

Movie Finder is the new demo app that ships with MyVector. It searches up to 1,035,695 TMDB movies by meaning, not keywords. One docker compose up gives you the whole thing: MySQL 9.7 with the MyVector component, a loader, and a small web app.

Try it live: https://demo.myvector.online/

We built it to answer the questions people ask us most. Does vector search in MySQL hold up at a million rows? Can I mix it with ordinary WHERE filters? What happens when I insert a new row? The demo shows the answers on the page, with the SQL that ran under every result.

Live: What you can do on the page

Every feature on the page maps to a MyVector capability, and every result panel shows its SQL.

  • Describe a movie in your own words and get the nearest films. Ask for “a boy wizard at a school of magic” and Harry Potter comes back first.
  • Filter by genre, year, rating and language. These are plain SQL predicates combined with the vector search, and the page tells you which filtered-search path ran and why.
  • Rank by similarity alone, or with a small boost for films many people have rated, so the famous match beats an obscure one at almost the same distance. Both are ordinary ORDER BY expressions.
  • More like this: a film’s stored vector becomes the next query, in pure SQL.
  • HNSW vs exact: run both side by side and compare speed and recall.
  • Add a movie: a plain INSERT, searchable within about a second, with no index rebuild. (Not on live demo)
  • Under the hood: live index details from myvector_index_status, a bar showing where each query’s time went, and a speed-vs-recall chart that sweeps ef_search from 10 to 640 against an exact scan.

How it works

The app embeds only your query text; every movie’s vector already sits in MySQL, so a search is one SQL round trip. The movie vectors come with the dataset, made by nomic-embed-text-v1.5 from each film’s title, tagline, and overview. The app uses the same model locally on the CPU, so there is no API key, and nothing leaves your machine.

New rows need no rebuild. Adding a movie is a plain INSERT; MyVector’s binlog listener picks it up and adds it to the HNSW index, usually within a second.

The SQL

The whole index is declared in a column comment. This is the movies table’s vector column:

embedding VARBINARY(3080) COMMENT
  'MYVECTOR COLUMN type=HNSW,dim=768,size=...,M=16,ef=100,dist=Cosine,online=Y,idcol=id,threads=N'

The loader inserts the rows, then builds the HNSW index with one call:

CALL mysql.myvector_index_build('movies.movies.embedding', 'id');

A search turns the query vector into a nearest-first list of ids with myvector_ann_set(), then joins back to the table. JSON_TABLE keeps the order:

SELECT m.title, myvector_distance(m.embedding, @q, 'Cosine') AS distance
FROM (SELECT myvector_ann_set('movies.movies.embedding', 'id', @q,
                              'nn=10,ef_search=100') AS js) src,
     JSON_TABLE(src.js, '$[*]' COLUMNS (rank_no FOR ORDINALITY, id INT PATH '$')) nn
JOIN movies.movies m ON m.id = nn.id
ORDER BY nn.rank_no;

“More like this” needs no embedding at all. It reads a film’s stored vector into the query variable:

SELECT embedding INTO @q FROM movies.movies WHERE id = @movie_id;

Filters pick one of two paths. The app counts each filter on its own index and takes the smallest count as an upper bound:

  • 50,000 matches or fewer: it passes the matching keys as the fifth argument of myvector_ann_set, so HNSW searches only among them.
  • More than that: it calls MYVECTOR_ANN_FILTERED, which takes the nearest candidates and keeps those that pass the filter.

Run it yourself

To run your own copy, from a clone of the repository:

cd examples/movie-finder
MOVIES=100k docker compose up      # then open http://localhost:8080

The first start downloads the TMDB data, about 7 GB, once. MOVIES picks how many films to load, most-voted first: 10k, 100k (the default) or full. Measured on a 16-core Arm (Neoverse-N1) Linux host with MySQL 9.7.2:

Step 10k 100k full (1,035,695)
Insert 7 s 41 s 407 s
Build the HNSW index (16 threads) 29 s 33 s 545 s
HNSW search 3–5 ms 4–7 ms 5–20 ms
Exact search (scans every row) 30 ms 280 ms 3 s warm
Recall@10, HNSW vs exact 100% 100% 100%

At a million movies, HNSW answers in 5–20 ms where a full scan takes 3 seconds, with the same top 10. Recall is for the unfiltered test query; a broad genre filter (Drama) at full size gave 80% at ef_search 100, and the page lets you raise ef_search to trade time for recall. For the full profile, give Docker about 12 GB of memory and set MYSQL_BUFFER_POOL=6G.

Running it on a remote server? The page and MySQL listen on localhost only. Forward the port instead of opening it: ssh -N -L 8080:127.0.0.1:8080 <server>.

Version note: the demo uses features newer than the v1.26.9 images: filtered search, MYVECTOR_ANN_FILTERED, per-query ef_search, and online updates that survive a restart. Until the next release ships, the README shows how to build a local image from main.

Data: the movie metadata and posters come from TMDB via a Hugging Face mirror. Your machine downloads it; it is not in the repository or any image. This is a non-commercial demo, not endorsed by TMDB.

Try it and tell us

Try the live demo first. To run your own, start with MOVIES=10k for a quick first look, then go to full to watch a million rows answer in milliseconds. The demo guide has screenshots and the other demos; the Movie Finder README has the full details.

If a query surprises you, or you want a feature the page doesn’t show yet, open an issue on GitHub. A star helps other MySQL users find the project.

Planet MySQL