Gil's LotD - Gil's Link of the Day: Always random, always interesting.

Hurting the feelings of the Chinese people

gil — Mon, 28 Sep 2026 13:48:31 +0000

Hurting the feelings of the Chinese people

Wikipedia has an in-depth article on the Chinese Communist Party's use of the phrase "hurting the feelings of the Chinese people" in their official communications. However, the graph on the page only runs until 2014, which is when Horng-luen Wang's paper from which it was sourced was published. What does the history look like since then? I asked Claude to come up with a way to grep the archives of the People's Daily and I have totals up to 2026-09-26.

First, our updated graph:

Second, here's Claude's explanation of its methodology:

When China objects to the act of a foreign government, it often says that the act "hurts the feelings of the Chinese people" (伤害中国人民感情). This project counts how often the phrase is in the People's Daily, the official newspaper of the Chinese Communist Party, from 1946 to today. The text of the paper is available: about two million articles, from a free data set up to 2025 and from the paper's website for 2026. Claude, an AI model, wrote the search and checked the results. The project owner does not read Chinese. This post tells how Claude found the phrase in these articles and how Claude checked the results.

Why a simple text search does not work

In Chinese, the base phrase has eight characters: 伤害中国人民感情. Word for word, it is "hurt – Chinese people – feelings" (伤害 – 中国人民 – 感情).

Written Chinese has no spaces between words. Also, writers often add words to the phrase or change its order. Some real examples, in English:

"hurt(past) the feelings of the Chinese people" (伤害了中国人民的感情). Chinese adds a small word to show the past.
"seriously hurt the feelings of the Chinese people" (严重伤害中国人民感情)
"hurt the feelings of the 1.3 billion Chinese people" (伤害了13亿中国人民的感情)
"hurt the national feelings of the Chinese people" (伤害中国人民的民族感情)
"hurt the feelings of the people of China and the Asian countries" (伤害了中国和亚洲各国人民的感情)
"the feelings of the Chinese people were hurt" (中国人民的感情受到伤害)

Each change adds or moves characters, so a search for the exact eight characters does not find these sentences. For 1989, a search for the exact phrase finds 15 articles. Claude's final search finds 38.

How the search works

The search does not look for one fixed text. It looks for a pattern: the parts of the phrase, and how far apart they can be. The pattern has three parts:

the word "hurt" (伤害)
a word for "the Chinese people" (中国人民)
the word "feelings" (感情)

The parts must be in this order. Other characters can be between them: up to 16 between "hurt" and "the Chinese people", and up to 20 between "the Chinese people" and "feelings". But a gap must not contain a comma, a full stop or a similar mark. This rule keeps the three parts in one clause. Thus the search does not join words from two different sentences.

The search also finds the passive form: "the feelings of the Chinese people were hurt" (中国人民的感情受到伤害).

For "the Chinese people", the search accepts a list of words with the same meaning. Examples are "the Chinese nation" (中华民族), "the Chinese" (中国人), "the people of our country" (我国人民), "the sons and daughters of China" (中华儿女) and "the descendants of the Yellow Emperor" (炎黄子孙), a traditional name for the Chinese. It also accepts "China" (中国) followed a short distance later by "people" (人民). This finds forms such as "the people of China and the Asian countries" (中国和亚洲各国人民) and "China's 1.2 billion people" (中国12亿人民).

Before the search, all spaces and line breaks are removed, because older texts often break a line in the middle of the phrase. The search accepts both forms of Chinese characters: the simplified form used in mainland China today, and the traditional form.

An article counts one time if it contains the phrase one or more times.

Warnings

Not every match is an accusation. Some are warnings, for example "do not again do things that hurt the feelings of the Chinese people" (不要再做伤害中国人民感情的事).

To help the reader find these, the search tags a match as a possible warning when words such as "do not" (不要), "avoid" (避免) or "never" (绝不) come before it in the same clause. The tag is only a guess from these words. It does not remove the article from the count: the count includes accusations and warnings. The project owner decided this, because a database search for the phrase also finds warnings. A reader who wants only accusations can remove the tagged articles.

How Claude checked the results by hand

The first version of the search was narrower. It accepted only two words for "the Chinese people", and it permitted a shorter gap. To find what it missed, Claude also kept a wider set of matches: all sentences with "hurt … feelings" (伤害……感情) and any word about China or the nation.

Claude read 814 matches, each with the text around it, in groups of 100 to 170. Claude gave each match one label and a short reason in English. The labels were:

accusation: the text says that an act hurts, or will hurt, the feelings of the Chinese people.
warning: the text tells someone not to do things that hurt these feelings.
denial: the speaker says that they will not hurt these feelings.
other group: the phrase is about a different group, for example the people of Taiwan and the mainland together, the peoples of Asian countries, ethnic minorities inside China, or a dispute inside China.

The check found three problems in the first version:

It missed 66 real accusations. Most used a word for "the Chinese people" that was not on the short list, or had a longer gap.
The "possible warning" tag was often wrong. It was put on words that start with "not" but do not make a warning, such as "not hesitate to" (不惜), "disregarding" (不顾) and "unfriendly" (不友好). It was also not put on most real warnings.
A second tag, for phrases about "the peoples of both countries" (两国人民), was put on unrelated words such as "relations between the two countries" (两国关系).

The second version of the search, described above, fixes these problems. On the matches from 1946 to September 2026:

It finds 498 of the 501 accusations that the check found (99%).
It finds 580 matches in total. 498 are accusations and 73 are warnings. The other 9 are denials or other uses.
Of the 76 warnings and denials, 68 have the "possible warning" tag.
2 real accusations also have the tag by mistake: a rhetorical question and an "if … then it would not" (如果……就不会) sentence. The search found these 2 accusations correctly; only the tag is wrong. Because the tag does not remove an article from the count, the count is correct.

These numbers are too good. Claude used the same matches to design the second version and to test it. A fair test needs new data: for example, a person who reads Chinese labels a random sample. No person who reads Chinese has checked Claude's labels yet. Also, the check did not include sentences with no word about China, such as "hurt our feelings" (伤害了我们的感情). Some of these can be accusations.

Wang's count

The graph in the Wikipedia article "Hurting the feelings of the Chinese people" comes from a 2014 paper by Wang Horng-luen of Academia Sinica in Taiwan. He searched a People's Daily database for the exact phrase and counted 319 articles from 1949 to 2013. The paper does not tell how the database search works, or if he removed some results by hand.

The first plan was to add new years to his series. For this, Claude's search had to give the same numbers as his. Claude tested four searches on 1989 and 2012. The numbers are articles in 1989 and in 2012:

The exact phrase (伤害中国人民感情): 15 and 5
"hurt", "Chinese people" and "feelings" in order, with short gaps: 35 and 12
"hurt" (伤害) anywhere, and "Chinese people('s) feelings" (中国人民(的)感情) anywhere: 29 and 14
"hurt", "Chinese people" and "feelings" anywhere in the article: 44 and 19
Wang: 28 and 27

No search gives both of his numbers. Thus the project did not add to his series. It makes a new series from 1946 to today, and shows his series only as a separate line.

The two counts compared

For 1949 to 2013, Claude's search finds 483 articles. If only the matches that the hand check found to be real accusations are counted, it finds 420. Wang found 319.

Both counts find the first use in the same article, on 17 December 1959.
Claude's count is higher than Wang's in 28 years and the same in 7 years.
Claude's count is lower in two years: 2009 (by 1 article) and 2012 (14 articles against his 27). Claude checked all 14 articles for 2012. All are real accusations.

The cause of the 2012 difference is not known. Possibly his database also includes the overseas edition of the paper, or it matches the words separately, or it has articles that are not in the copy of the paper that this project uses.

gil Mon, 09/28/2026 - 9:48am

6/22/26 Gil's LotD: Finding the Best Dog Treat with Statistics

gil — Tue, 23 Jun 2026 00:21:23 +0000

6/22/26 Gil's LotD: Finding the Best Dog Treat with Statistics

https://www.wespiser.com/posts/2026-06-19-best-dog-treat.html How to find the globally best dog treat from head-to-head comparisons.

gil Mon, 06/22/2026 - 8:21pm

AI Predictions

gil — Sun, 10 May 2026 23:14:47 +0000

AI Predictions

I'm putting these down on paper, let's see how I do:

Model intelligence has plateaued. The scaling hypothesis was rejected years ago, but labs and true believers keep it around like a ghost to keep people hopeful. We've mostly seen gains from improved tooling and chain of thought techniques. Chain of thought is useful but it shouldn't be counted as improved model intelligence: if you had the ability to make a model that performed as well as a CoT one without spending the extra tokens you'd just do that.

For industry finances we need to wait for the GAAP numbers. Believe no one, trust nothing until we get something audited (or at least something that carries some liability) out to the public. This could be a big moment: it probably won't deflate the bubble but it could be a serious shock of cold water. I'm sure all the labs are hurtling along to IPO as soon as they can but I also wonder if there's a bit of a game of chicken around who has to be the first to bare it all.

I don't think labs are profitable on marginal inference. At the least they're just barely breaking even.

Model training depreciation is going to be steep, I think the realistic way to model it is to depreciate it in just a year[1] or year and a half. With time this race will slow down and that'll stretch to two years, maybe three.

Anthropic may be compute constrained, but they're also probably unprofitable: you don't look at killing the $20 plan for code use unless you're not getting the profit you need out of it. If you were profitable on compute you'd welcome signups even when you're compute constrained: your profit only increases if users pay but can't use your system! I'm aware you need to treat profitability on these monthly plans in a more complex way: paying but infrequent users wind up subsidizing users who saturate their usage. Your customer growth can happen with either cohort too: it's great for your business if your rate of gain of subsidizers beats the rate of gain of maximizers. But decisions like dropping features from the $20 plan, killing Sora, and other comments made by Altman suggests that labs have seen growth in revenue but it's from the smaller margin customers.

OpenAI is doing everything sleazy and uncompetitive that they can to win the market. Right now they've been raining down compute on codex users in hopes of starving out competition so they can extract monopoly rents later. Altman is burning as much cash as they can to get any edge they can. Some of this might even be uncompetitive: I am sure that they have internal predictions to justify their agreements to buy compute and drive up its price across the market (see RAM prices) but I am also thinking that if OpenAI trips up at all along the way there they'll breach any agreements and free up a lot of capacity very quickly and I think it is more likely that we'll see those breaches than not. Once again, we have to wait to see the GAAP numbers and cashflow.

[1] I'm fully aware how silly it is to call it depreciation when your asset has a useful lifespan of one year but things are moving fast.

Update on September 13th: The frontier labs are discussing a pause on AI development for safety reasons. There are a lot of ways this can be read, I won't expand on that. Sam Altman also confirms that the OpenAI IPO is delayed until next year; previously, it was expected to come in the fall of this year. Generally, if your business is a winner, it can IPO. You can't IPO overnight but you can do so pretty quickly if you need to. It's the real stinkers, companies like SpaceX, that stay private for decades, missing multiple boom and bust cycles. Anthropic's kept up its profit-squeezing antics, keeping token costs high while the entire industry involutes. Anthropic is also slow rolling a cut to plan limits this month. The latest models spew incomprehensibly verbose English and fill code with useless comments, boosting token spend on the write and on every single read afterwards. The proposed pause frees up model training compute for consumer compute, shifting expenses from R&D to COGS. This isn't necessarily a cost-cutting measure because it enables them to scale up to new customers and deliver more compute to existing customers, all good things for the income statement. But it's hard to argue that this isn't at least a little bit of an air bubble in the cash flow that goes from frontier labs to hardware purchases and the datacenter buildout pipeline. I don't think they'll pause buildouts, there's still plenty of demand for compute, but even a little skip in the incestuous cash flows moving between hardware suppliers could cause some deleveraging.

gil Sun, 05/10/2026 - 7:14pm

Why did the United States invade Iraq?

gil — Sun, 04 Jan 2026 14:01:39 +0000

Why did the United States invade Iraq?

Kitty Saddam Hussein

There's a half-remembered folk history of the lead up to the invasion of Iraq that's shaped the war's modern understanding in the United States. Here, I'll lay out in brief the several contributing factors that made the invasion happen.[1]

They Thought It Would Work

This is sort of silly to bring up because it applies to most everything that has ever happened in history: people do things because they think they'll work out, the risks will outweigh the benefits, and in the end they'll be better off than how they started. But I want to discuss this here because the modern discourse isn't always interested in understanding the perspective of the perpetrators. And it's not just Bush and Cheney that thought it would work: the war happened because there were multiple constituencies or interest groups who thought it would work and these groups drew their own benefits, often different from each other, from the war. The Republican party, just like the Democratic party, is pluralist: there are multiple groups with their own interests and can have different motivations for the same action. That's mostly what we'll discuss here: who are those groups and what did they see in Iraq? Ultimately, what made the US invade Iraq was this confluence of interest groups all seeing an advantage to what Dubya was proposing.

The Same Team Already Did It Once

They thought it would work because the US already tried invading Iraq once before in the Gulf War and that went pretty well. Some of the same people came back to do it the second time, notably Dick Cheney (SecDef) and Colin Powell. Maybe this mostly reassured the administration itself but you can see how they might go into the war with the mindset that it'll work out just like the Gulf War and it'll be an easy publicity win.

Cultural Conservatism

Remember that Republicans are genuinely conservative: they hold conservative political beliefs, they think those beliefs are consistent and correct, they want to see a country and world oriented to their conservative views and they actively work towards those goals. I want to dispel the sometimes implicitly held idea that these beliefs are purely rhetorical, just a fig leaf over someone's genuine political motivations. Maybe this does not describe all Republicans but it describes lots of them, especially once you get away from politicians and down towards the base. For the duration of the Cold War these conservatives had a consistent message: the US stands against Communism and authoritarian governments. An invasion of Iraq gives a renewed version of the branding: the US still stands against authoritarian and oppressive governments and is willing to use its industrial and military might to be a force for good in the world. It gives the masses something concrete to believe in and a movement to belong to. A war reaches all corners of the society, it's discussed in mass media and the public discourse, it gives an opportunity to rally around the flag in public, and it can even reach down into the smallest rural communities when their young men enlist, potentially giving a personal connection to the far-away conflict. The war becomes a way to build the conservative culture that you wish the country had.

Saddam Was A Bad Dude

The moral aspect of overthrowing Saddam resonated with the public because there was a sort of cosmic background radiation about the guy from the events of the 80s and 90s. Saddam had use chemical weapons against his own people before! He attempted genocide against his own citizens in 1998. Even Bernie Sanders, who famously opposed the 2003 invasion of Iraq was in favor of overthrowing Saddam and destroying his ability to manufacture WMDs in a speech in 1998. Bernie is not being inconsistent here[2]: he believes the US should defend the oppressed people of the world, he just differs on the details of how it should be done. In the 1998 speech he's arguing the US should be arming an anti-Saddam insurgency and in the 2003 speech the US should operate within a UN coalition instead of unilaterally. As the Iraq war dragged on and the arguments made by the Bush administration for the 2003 invasion flopped that still wasn't enough to shake popular support of the invasion because of this pre-positioning of the populace to accept that overthrowing Saddam was just.

Stovepiping

We now know that the evidence presented for Saddam's WMD program or collaboration with Al-Qaeda was wrong. Today there is a common belief, either stated or implicitly held, that the entire government intelligence community was behind this evidence and Iraq was the big reveal of the intelligence community's incompetence. In other words: they got it wrong with Iraq, why would you trust them on the issues of today? But even at the time there was widespread criticism of the Bush administration's evidence both outside and inside of the government: for example, the Plame affair can only happen as an act of revenge if someone is credibly and publicly criticizing the government's position. The administration had to turn to tricks like the Office of Special Plans, a whole institutional apparatus created just to shortcut the existing intelligence community to give Bush the evidence he wanted for the invasion. This stovepiping technique is insidious: it hijacks the credibility of the existing organization and uses it to sway those who trust the institutions with a work product ultimately not created by the institutions. Personally, I think stovepiping contributed to war support from a lot of people who "should have known better" like the Democratic leadership at the time: they were victim to an intentional plan to take advantage of institutional trust. It would have been better if they leaned more into the non-stovepiped arguments that were made but unfortunately that didn't happen.

The End Of History

There was a well-timed argument made in 1989 that liberal democracy won out over authoritarian governments and that going forward the authoritarian governments of the world would eventually collapse out of the popular preference for democracy. Francis Fukuyama, the author of this "End of History" argument, eventually falls in with the like-minded 90s think tank Project for the New American Century. The PNAC becomes the spawning ground of neoconservative thought and Bush drafts many of its participants into his administration. That's the history, but I want to convey some of the mindset of these neoconservatives. Coming into the 90s they were greeted with a validation of their belief in liberal democracy and capitalism with the collapse of the USSR. Under George H.W. Bush the Gulf War was an incredible success, once again validating their faith in the US military. As the 90s dragged on the economy took off, eventually becoming the dot com bubble but genuinely transforming business and society for the better with improved computer and communication technology. The PNAC was high on its own supply, seemingly validated at every turn, and had no problem pitching aggressive and risky plans like the overthrow of Saddam Hussein. The neoconservatives were the constituency of fart-sniffing true believers.

Shortly after 9/11 George W. Bush gives a speech where he famously pitches that Al Qaeda "hates us for our freedoms". I want to bring this up because I think Bush is actually making an "End of History" argument here in a more digestible form for the public. The claim on its face is not true, 9/11 and Al Qaeda were motivated by Israel, but Bush's statement presupposes the End of History desire for liberal democracy and turns it into the motivating force for Al Qaeda. Here, the neoconservative ideology gives a way for the the administration to sell its ideas to the public. The same thing happens with "we will be greeted as liberators" - once Saddam is removed the End of History-driven desire guarantees the Iraqi people will self-organize a democracy.

Defense Contractors

A popular idea at the time was that defense contractors, looking for government war contracts, demanded the invasion. I think the military-industrial complex is certainly one of the groups in the coalition in favor of war but the popular conception that the defense industry dictates what it wants out of the government is backwards. I will elide my argument here for a later post but I think the appropriations and bidding processes adequately keep civilian control over contractors and their profit.

Oil

Oil is important but I don't think the administration was motivated to invade Iraq for its oil. Iraq didn't have the oil, Kuwait did! That's why Iraq invaded Kuwait in the first Gulf War after all: Iraq's oil industry was economically uncompetitive. The US also started its invasion with the destruction of Iraq's oil industry infrastructure which is not what you'd do if you want to start exporting their oil. Dick Cheney, immediately before he was Vice President, was CEO of Halliburton, a company that originally provided oil infrastructure services but moved into defense contracting. Halliburton did secure oil and defense contracts from the US government during the war and Cheney and Halliburton's other stockholders profited from it. War profit certainly motivated some Republicans but, as I've laid out so far, there are many constituencies that have to come on board and this only motivates one of them. Cheney, in particular, I think is largely motivated by his neocon ideology even though he also stands to profit substantially from the war.

Revenge?

Iraq did attempt to assassinate George H.W. Bush, the father of George W. Bush, and some have floated the idea that the invasion of Iraq was motivated by GWB exacting revenge on Saddam for the attempt. Maybe? If could be in this big pile of motivations. But even if it was, it motivates only one person (albeit a very important person): GWB himself. Maybe it helped grease the wheels a bit but I don't think it contributed much.

Israel

Saddam Hussein had no way to strike the United States: he has no intercontinental bombers, no international navy and no ICBMs. Despite what the Bush administration suggested in the lead-in to the war Saddam had no capacity to attack the domestic United States. Israel, on the other hand, is a closer target and Iraqi-Israeli relations are not great. After the first Gulf War the UN imposed strict limitations on Saddam's missile program, limiting him to missiles unable to strike Israel, and these limits were enforced by UN inspections for some time. But it is not unreasonable to suspect that one day Saddam might acquire missile technology with the range he needs to hit Israel. At the same time, Israel fears a war against all of its neighbors all at the same time (not that strange of a hypothetical, it's happened before!) and an Iraqi military with its population, relative military competence, and missiles that can strike from Iraq make it a serious threat. If Saddam does resurrect a chemical, biological or nuclear program with the right missiles he could launch dirty bombs at Israel. You can see how Saddam Hussein and his military are a much larger threat to Israel than they are to the United States. A war that both overthrows Saddam, destroys his army and destroys any WMD labs or missile capability motivates the pro-Israel constituency in the United States in favor of the war.

Al-Qaeda opposes the United States because they oppose Israel; they see United States as the force that keeps Israel alive and attack the US in an attempt to make Israel vulnerable. If you view the conflict through Al-Qaeda's framing the Bush decision to invade Iraq is part of the revenge for 9/11: there is a grand battle to wipe Israel from the map, should you attack Israel's allies the US will strike one of Israel's enemies to keep Israel with an upper hand. When the Bush administration first made its argument to the public for Iraq it tried to pitch it as revenge for 9/11 but that message fell flat. The WMD argument was introduced later as a pivot away from the 9/11 argument. Ultimately, the 9/11 argument didn't work because the broader population didn't share Al-Qaeda's grand battle framing of 9/11 and Israel, however, given that the administration did try and present it this way at the beginning reveals that at least the guys at the top see the battlefield in this way.

Kitty History

Everything I wrote above is bullshit. Here's why it really happened.

[1] Shoutout to Why Did We Invade Iraq? by Charles Davis which argues basically the same thing I am doing here but a bit more thoroughly and rigorously. I found this article while doing research for this one.

[2] My read on Bernie's 1998 speech is that he really wants to pursue his isolationist goal. Bernie is opposed to all US intervention because he wants to scale back US influence and create a multipolar world order. In this speech he is trotting out these arguments to make the "of course I believe in X, I just have problems with the particular details of X in this instance" style of argument to object to the war in an unobjectionable way.

gil Sun, 01/04/2026 - 9:01am

Copying my VPS to a new disk

gil — Wed, 27 Aug 2025 15:44:38 +0000

Copying my VPS to a new disk

I've been a customer of TornadoVPS since 2011. The VPS came with an ext3 formatted filesystem which was at the time the default filesystem in the Debian installer. In the meantime the kernel's ext3 driver depreciated and eventually removed prompting me to do the in-place upgrade to ext4. However, at some point I wanted to get a new, clean filesystem with bigger inodes, fast_commit support and year 2038 fixes. Here's how I copied the filesystem over to a new ext4 partition.

The new disk

TornadoVPS support was willing to attach a second block device to my VPS to support my migration with the understanding that I would turn the original disk over to them to be deleted when I was done copying my data over. The new block device was hotplugged into the VM and was ready to be formatted right away:
fdisk /dev/xvdb mkfs.ext4 /dev/xvdb1 tune2fs -O fast_commit /dev/xvdb1

There are plenty of useful ways to set up fdisk, just don't forget to have partitions aligned and give some space at the start of the disk for grub's data. (That is, have the start of the first partition not be right at the start of the disk.)

Backups

I rebooted into a rescue system to do the off-site backup of the original disk. You could use dd to dump the whole thing but there's a nifty e2image command that just dumps the blocks used by the filesystem. You can dump it over SSH easily:

ssh -i ~/.ssh/id_xyz root@the_rescue_system e2image -apr /dev/xvda1 - > backup_YYYYMMDD.img

To restore the image, you'd do it in reverse:
cat backup_YYYYMMDD.img | ssh -i ~/.ssh/id_xyz root@the_rescue_system e2image -apr - /dev/xvda1 backup_YYYYMMDD.img

Copying everything over

I rebooted into a system-provided rescue disk to do the copy although you may be able to do this from the initrd shell or single user mode. I didn't want to be running anything off of the disk, even mounted read-only, just to be safe.

mkdir /mnt/disk1 mkdir /mnt/disk2 mount /dev/xvda1 /mnt/disk1 mount /dev/xvdb1 /mnt/disk2 time rsync -ahHAXxS --info=progress2 --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} /mnt/disk1/ /mnt/disk2

In brief, the flags to rsync are:

--info=progress2: instead of --verbose, this is a less spammy and more useful output for copying a large number of files.
--exclude: A sensible list of directories that don't need to make the jump.
-a: archive. (preserve timestamps and owenrship)
-h: file sizes in human-readable units
-H: preserve hard links
-A: preserve ACLs
-X: preserve xattrs
-x: don't cross file system boundaries. Unlikely to affect anything but a safe default.
-S: preserve sparse files

Fixing up the bootloader

First, you need to grab the UUID from the new filesystem.

tune2fs -l /dev/xvdb1 | grep 'Filesystem UUID'

Update the /etc/fstab on the new disk with the new filesystem's UUID.

Next, you want to start configuring the bootloader. First, change the GRUB_TIMEOUT in /etc/default/grub to -1 to let grub halt and let you choose the new disk. On the original disk's /boot/grub/grub.cfg I found the menuentry section for my current kernel, copied it over, updated it with the new UUID and renamed it. This was to support booting from the first disk onto the second as it is not possible to configure the VPS to boot from the second attached disk.

If you manage to get grub installed to the MBR of the second disk you could instead have the first disk chainload grub via a simpler menuentry:
menuentry "Boot from second disk" { set root=(hd1) drivemap -s hd0 hd1 chainloader +1 }

Once you're done editing the files you need to rebuild the grub configuration on the first disk. The easiest way to get a working chroot is to use the new systemd-nspawn tool, however this requires you to have the systemd-container package installed. It's a quick dependency to install and you should be able to do it with the package manager shipped in the rescue system.

systemd-nspawn -D disk1 -a --bind=/dev/xvda --bind=/dev/xvda1 --bind=/dev/xvdb --bind=/dev/xvdb1 --resolv-conf=off /bin/bash (in the chroot) update-grub

Although you don't need the xvda / xvda1 bind mounted into the disk for this here is how you'd do it. You almost certainly want the --resolv-conf=off to avoid clobbering your resolv.conf. Note that you may be tempted to chroot into the second disk to run grub-install to the second disk (/dev/xvdb), however, the bind mounts were confusing grub and I had no luck with it.

Finally, outside of all the chroots, unmount your disks and restart.
umount /mnt/disk1 umount /mnt/disk2 shutdown -r now

Finishing up the new system

Upon reboot you should be able to choose your new menu option and boot off of the filesystem of the new disk. Check the dmesg after booting for the new disk's UUID to make sure you've come up on the right system! If everything is looking fine, you can install grub to the MBR of then new disk:

grub-install /dev/xvdb update-grub

And then do a reboot to make sure everything's still working. if you added the chainloader menuoption to the first disk's grub.cfg above you can try that to make sure the MBR installation of grub is correct.

Post-install

I think the -S flag to rsync turned my on-filesystem swap file into a sparse file. Linux didn't like that and I had to create a new swap file and attach that.

Once everything was up and running on the new system I powered the system down and asked support to detach the old volume so I could do a clean start onto the new disk for the first time.

Conclusion

This process was easy enough I would not be opposed to re-creating filesystems after major kernel updates, especially if there are new filesystem features or flags that I can enable on the new code.

gil Wed, 08/27/2025 - 11:44am

2/2/25 Gil's LotD: McDowell Nature Preserve

gil — Mon, 03 Feb 2025 01:08:51 +0000

2/2/25 Gil's LotD: McDowell Nature Preserve

https://en.wikipedia.org/wiki/McDowell_Nature_Preserve The McDowell Nature Preserve is an urban park with hiking trails in Charlotte, North Carolina.

gil Sun, 02/02/2025 - 8:08pm

5/24/24 Gil's LotD: Big Rock Nature Preserve

gil — Fri, 24 May 2024 12:27:30 +0000

5/24/24 Gil's LotD: Big Rock Nature Preserve

https://en.wikipedia.org/wiki/Big_Rock_Nature_Preserve The Big Rock Nature Preserve is a park known for its "very big and quite significant rocks."

gil Fri, 05/24/2024 - 8:27am

Taylor Swift albums as Linux distributions (Manjaro Linux will surprise you!)

gil — Wed, 27 Sep 2023 23:53:38 +0000

Taylor Swift albums as Linux distributions (Manjaro Linux will surprise you!)

Taylor Swift s/t - Amazon Linux 2

Taylor's self-titled release came out in 2006. As an unknown talent she had to spend several years prior practicing her songwriting and slowly putting together what would be a chart-topping debut album. Everyone's familiar with it, most enjoy it, but it's rarely someone's favorite. Amazon Linux 2 is a fork of RHEL, a distribution with its own lengthy history, and is the culmination of effort in and outside of Amazon. Nearly everyone's worked with Amazon Linux at some point and sure, it gets the job done, but nobody really reaches for Amazon Linux outside of AWS or can say it's their favorite.

Fearless - Maemo

Taylor hit the typical sophomore slump with Fearless. It was time to bring something new to the table and she just didn't quite pull it off. The result was forgettable, and so is Maemo, Nokia's attempt to break the iPhone/Android duopoly. Did you know that after the Maemo/Moblin merge an open source community kept maintaining the original Maemo codebase? Nobody does because nobody used this stuff. It was a desperate attempt to stay relevant and a cash grab.

Fearless (Taylor's Version) - Tizen

Speaking of desperate cash grabs, Taylor Swift re-recorded and released Fearless to turn her one-time album purchases into ARR (Taylor's Version). Tizen was spun up from the ashes of MeeGo (the child of Maemo and Moblin) by Intel and Samsung. Intel wanted to get in on the smartphone revolution and Samsung wanted a knife in their back pocket should Google threaten their independence but in the end nobody really got anything out of it.

Speak Now - NixOS

Taylor's third release demanded creativity and our girl delivered on Speak Now with the start of her turn from country to straight pop music. NixOS also struck out for new territory. They've invented creative ways to package software and deliver on an immutable, declarative and transactional system but the usability of the system and the overgrown DSL used to package it is still an open question.

Speak Now (Taylor's Version) - Guix

Nobody requested a re-release of Speak Now but Taylor did it anyway. Nobody asked to wrap Nix with Guile Scheme and use it to build an entire distribution but someone at GNU did it anyway.

Red - Arch Linux

No, it's not Red Hat, even though she's literally wearing a red hat. Red is the fan's album, it doesn't introduce anything new but instead carves out a comfortable, intimate space for her listeners. The Swifties love it, fans often rank it as their favorite, but it has little weight out in the greater world. Arch Linux has passionate fans all over the web but at the end of the day it's all but missing from the server room.

Red (Taylor's Version) - Manjaro

It's a fork of Arch. Nobody asked for either of these and yet here we are.

1989 - Android

Taylor finished her pivot to pop music with 1989 and put out her most popular and accessible album yet. User friendliness and accessibility is a must if you want to capture the broadest audiences. Google created a Linux with working sound out of the box, a useful sandboxing model for user applications and a coherent user experience but only after throwing out the entire Linux userspace. As a reward for their efforts Android is the world's most popular software platform and Linux's world domination is complete.

Reputation - Ubuntu

Blaccent Taylor is an embarrassment and simply shouldn't exist. Ads in /etc/motd are an embarrassment and Ubuntu shouldn't exist. Bonus: both embraced bold, unconventional color schemes.

Lover - Red Hat Enterprise Linux

Lover is another pop album, doing a little bit of everything and has broad appeal. It turned into a sleeper hit years after its release when Cruel Summer started charting again in 2023. RHEL's been around forever, it's used for everything, it has its fans and it simply isn't going away.

Folklore - Slackware

Folklore is something new and stylistically unlike any of her previously releases but also her attempt at finding a nostalgia that could resonate with her now diverse fanbase. Slackware's an OG, older than everything on this list, but even back then it was nostalgic for the commercial UNIX workstations of the 80s.

Evermore - Gentoo

Evermore came quickly on the heels of Folklore and kept its theme, writing about the timeless characters of the past and the way things used to be. However it wore out its welcome and wasn't nearly as well-received as Folklore was. Gentoo had its moment in the sun in the 2000s but you don't seem to hear much about it anymore. The nostalgia for building everything from source, via shell scripts, on the dinky one and two core machines of yesteryear may have inspired ebuild but we've moved on.

Midnights - Debian

Taylor Swift focused on what really matters in Midnights: accessible pop music, TikTok-friendly hooks (nice!) and simple, reliable operating systems found on every server and in every container across the globe.

1989 (Taylor's Version) - ???

Who knows? Maybe it'll be your favorite distro, maybe it'll be one of those Android vendor kernels that hacked in massive security vulnerabilities to ship some questionable junkware feature.

gil Wed, 09/27/2023 - 7:53pm

The RESTRICT Act

gil — Thu, 30 Mar 2023 17:22:11 +0000

The RESTRICT Act

Banning or regulating TikTok has been discussed for years but now Congress may actually make a move towards it with the RESTRICT Act. Lawfare has the history of TikTok regulations and a brief summary of the RESTRICT Act and gives some useful analysis, although I wish they'd write more on it given the popular attention the bill's had so far. Vox also has a summary without a lot of analysis. I'd like to talk through my analysis of the issue and give some of my perspective.

The Story So Far

TikTok is different in important and interesting ways. Taking a step back in American history, and looking at the bigger picture of American media, we've always had media that is, at a fundamental level, at odds with the state. You don't sell newspapers with happy headlines. But at the same time, also at a fundamental level, all American media has also been aligned with the state's interests. This is a tension but not a contradiction. The state wants to propagate itself and preserve its influence. The media wants the same for itself, but it is also reliant on the state as they are part of a shared American society. Should the state's existence or influence be threatened the media's existence or influence is also threatened. Both the state and the media are, at some level, in alignment because they are both comprised of Americans sharing an American society and all the popular media we've seen so far in American history has shared this alignment.

There is also an overlooked view of the relationship of the state to its subjects, one that most often arises between the state and corporations. The primary relationship between the state its subjects is the one where the state gives direction, in the form of law, and enforces the law. It is the relationship where the state is superior to its subjects. In the case of the media, the First Amendment puts strong restrictions on the state's power in this relationship by restricting the laws the state can pass. But, in a case that is often overlooked, the relationship between the state and one of its subjects can also be one of equals. This typically looks like a business deal or any place where the state might sign a contract. In those situations the state and its subject are working together: both have their own agency, interests and goals and both are negotiating to find a mutually beneficial outcome. This is a form of soft power that the state can wield to accomplish its goals. Some people are uncomfortable with this soft power, they think the only way the state should ever exert power is in the explicit style of the first kind of relationship. But this soft power is inevitable and those who would wish it gone are naive. It arises because the the state and its subjects are not one and the same, a distinction that comes from liberalism itself and does not exist in authoritarian states. The state, enforcing a liberal order, has distinguished itself from its subjects, giving them agency, and this separation creates the opportunity for soft power when they happen to interact as equals.

The Issue At Hand

Those are some useful ideas but also abstract ones. They could apply to TikTok but they also apply to any number of apps, mass media platforms, or specialist products like SolarWinds. However, the topic at hand is much more concrete: how could TikTok be used to further the objectives of the Chinese state? Let's walk through some examples of the levers available to them through TikTok.

First, the most extreme and most unlikely thing China could do with TikTok is to push zero days to everyone's phones. This is unlikely because it is an act of war, and if it did ever happen we'd have more important things to worry about. Cybersecurity is still important, we should promote security practices in the personal, private and public spheres to protect us from realistic cybersecurity threats. But even in the context of a hot war a digital Pearl Harbor is not necessarily a great idea. It is most beneficial for China if the general population's view is "what did China ever do to us?" and not "those bastards hacked everyone's phones and screwed everything up!" This is a case where soft power can be more useful than hard power, an important idea I will come back to later.

Next is the collection of user data. This includes the data mediated by the phone's security model (the permissions you've granted the app) but also your own interactions with the app. For TikTok, that would include your interests (based on the videos you've watched and interacted with) and any social graph you've built in the app (any phone contacts you've imported, any friends you've added on the platform, any DMs you've sent to friends). That is a lot of very personal information! I am sympathetic to people who want to take a broad brush across the industry to reduce the total amount of information any app is allowed to collect on a user or reduce the legal uses of that information. If the current baseline of restrictions is unacceptable, let's raise that baseline.

Even if we find the baseline is acceptable the consumers of that data are different actors with different goals. The technical and legal baseline is just a baseline and it is not unreasonable to have different, more stringent standards for different users of the same data. Facebook and Google collect this data because they want to sell you ads, and TikTok does as well. But the Chinese state has its own interests and should they find TikTok or its data useful to promote them they have TikTok at their disposal to do so. TikTok is different, in ways we haven't seen before in American history, and it should not surprise us if it is subjected to a higher legal standard. We can't reason about it in the exact same way we'd reason about the American media and it is not unfair or illiberal to treat it different than the way we've treated the American media.

Getting to specifics, what is some of this soft power that China could exercise through TikTok in its role as part of the media? Remember that in 2020, a video of the execution of George Floyd went viral on the internet, reaching every part of American society. We saw race riots in several large cities, but don't overlook that we also saw civil rights protests happen basically everywhere. There were civil rights protests in Vidor, Texas - a place that is arguably still a sundown town! The simple ability to promote a strategically chosen video to the American public is a soft power. It is a tricky power to use but when the stars align it can have vast influence over the entire American society. TikTok has this soft power and unlike the media of the past it is not broadly aligned with the American state. TikTok is different.

As an aside, I am not saying anything about the specifics of the George Floyd video, but instead I think it is a familiar and modern example of the strength of the soft power of viral media. In a more concrete example, TikTok could promote English-language videos of the Chinese-picked leadership in Tibet and Tibetan Buddhism, popularizing the idea of an aligned Tibet and covering up China's colonization of Tibet. You and I may know that Tibet was invaded for geopolitical reasons but it's much easier for the public to digest videos of a happy and unified Tibet over content that explains snoozefest details like the early security objectives of the People's Republic of China. And even in this example it would only be under the scope of the RESTRICT Act if it was done to steer American decision-making on a topic.

This is also an opportunity for the state-and-media-as-equals soft power introduced earlier. Fundamentally, the First Amendment prevents the state from telling the media what to do. Coincidentally, if the colonization of Tibet results in unhappy headlines, the media might print it anyway. When the state and media interact as equals, the state now has an opportunity to bring attention to stories not covered by the media. But it could also bring forward its intelligence or analysis on what the media has or hasn't covered and the media, still acting with its own agency, can run or bury stories in its own response to what it's learned or negotiated.

The question at hand is not one of liberalism or to the degree that our government should set a baseline of liberalism for the media at large. Liberalism is what created this soft power and got us into this situation in the first place! We are instead looking at the paradox of freedom (see: the large body of philosophy on this topic) and the question is if American state should be allowed to defend itself and its interests against the narrow case of specific organizations that are fundamentally unaligned with itself. I think it does, and I hope you agree with me - liberalism doesn't uphold itself, it has to be defended. That is what the RESTRICT Act sets out to do, and I think it does an OK job at it.

What does the RESTRICT Act do?

If you really want to know, go read the bill or go read the Lawfare article. But in short, the RESTRICT Act doesn't call out TikTok by name. Instead, it seems to share the same ideological framework I've laid out here: it defines a category of organizations that are fundamentally unaligned with the American state, who at times could wield soft or hard power against the United States or its citizens, and it delegates identifying the specifics of the abuse of that power and the state's response to it to the executive branch. This sort of delegation is typical and expected under our Constitution, and this new power comes with additional protections: it can only be exercised against information technology systems and it limits that further to systems under the control of foreign adversaries. The scope is narrow enough to make it consistent with my understanding of liberalism and I'd go as far as saying it's consistent with all but the most naive conceptions of liberalism.

Once you get past the bill's scoping of covered transactions the powers delegated to the executive branch are intentionally broad. The purpose here is to give the government flexibility beyond prohibition ("you can no longer do this thing") as a response, which gives the government flexibility to scope its own exercise of power against the covered transaction. I am sympathetic to criticisms that the law could add explicit scoping of delegated powers here but understand that it may be hard to effectively draft it. The already explicitly-stated prohibition is the most impactful of powers the government could exercise and it seems like a necessary power to let this bill have any effect at all. There is little point in limiting the seemingly-broad "any other action" when prohibition is going to be a necessary part of the bill anyway. However, I admit that I could just lack imagination on a better solution.

gil Thu, 03/30/2023 - 1:22pm

Features I'd like in PostgreSQL

gil — Sat, 28 Jan 2023 17:51:24 +0000

Features I'd like in PostgreSQL

I’ve put in my fair share of time with PostgreSQL. It has no shortage of functionality and I believe its expansive feature set drove a lot of its growing popularity in the 2010s. As my time with the database has worn on I’ve come up with a handful of features that I wish the database had. Some of these are ideas borrowed from other systems, some are inventions of my own. The real tragedy is that I simply don’t have the time to implement the patches myself.

–i-am-a-dummy mode

This idea comes from MySQL whose stock CLI client supports an –i-am-a-dummy flag. When enabled, MySQL identifies any UPDATE or DELETE that lacks a WHERE clause and fails them with an error. In other words, if you intended to DELETE FROM table1 WHERE pk = 10, started typing it into your prompt, and only got as far as DELETE FROM table1 before accidentally executing the query, the query errors out instead of having every row deleted. More modern versions of MySQL also use the flag to block queries that return excessively large result sizes, require the WHERE to reference a key column, or allow a LIMIT in place of a WHERE.

Update: User felixyz on lobste.rs points out that there is a pg_safeupdate server-side extension that blocks UPDATE/DELETE without WHERE.

Update: Andrey Borodin and Nikolay Samokhvalov of Postgres TV streamed themselves implementing this behind a GUC! They sent in the patch but it was rejected by Tom Lane.

Unit test mode (random result sorting)

The order of a query’s results is famously unspecified unless you nail it down with an ORDER BY. The flexibility is useful: sometimes you need a stable order for whatever code consumes the query results and sometimes you just don’t. But there is a secret, third thing: some code comes to accidentally depend on a coincidental ordering of the results. To help flush these out it would be neat if PostgreSQL could intentionally shuffle the order of query results if a given query did not specify an ORDER BY. I imagine this as a run-time configuration field for the database that you’d turn on when running automated tests or in a testing environment. The alternative, folding every query in your application into a subquery that randomly sorts the results, is ugly and invasive, although perhaps some ORMs could implement the feature at their level.

Update: User avianlyric on Hacker News pointed out that CLUSTER can can be used to reorder results in a pinch.

Query progress in psql

This is another feature from MariaDB. It’s possible in both databases to query the database to get progress information on several types of queries, but the ALTER TABLE, CREATE INDEX, DROP INDEX and LOAD DATA INFILE queries also automatically inform the CLI client on their progress. The CLI uses this to draw a nice animated status bar with some informational messages. PostgreSQL lets you query the system catalog for the progress of several query types but has no live update functionality in the psql client.

Pandas-like join validation

I am a true database expert who has never screwed up a join before. But there are many beginners out there and it would be neat to extend the SQL language to support validation features like Pandas has for its join function. When doing a join you’d specify what relation you expect to have unique keys and which one has potential duplicates and PostgreSQL could warn or panic when this assertion is triggered. Some hypothetical syntax:

SELECT x, y
FROM t1
JOIN t2 USING (key) VALIDATE 1:m

Which would have PostgreSQL check that t1.key has no duplicate rows and errors if it comes across any. This is another place where it would be useful to have a runtime option to disable the expensive uniqueness checks.

Pandas’ “suffixes” argument, which automatically renames overlapping columns in the output, and “indicator”, which emits a new column identifying the left or right table or both as the source of an output row, are also neat ideas that seem really helpful for relational newbies trying to troubleshoot their joins. Pandas is insanely popular, it’s taught to a lot of very inexperienced developers, and I think features like these were earned with newbie blood. I suspect PostgreSQL’s users and developers don’t get that sort of exposure to newbies and their typical problems but there’s no reason why they can’t learn from the hard-earned experience of Pandas and incorporate some good ideas.

Update: User richbell on Hacker News brought up DuckDB's Friendlier SQL which is a superset of sensible extensions to SQL. I endorse all of these ideas except for the trailing comma fix - join the dark side and use leading commas!

JIT support for CREATE INDEX

Today, the JIT support in PostgreSQL is closely tied to query planning and execution. It rests upon the object-oriented nature of the execution plan and emits LLVM IR for each plan node as it traverses the plan tree. It’s an elegant technical approach, and for its sweet spot of long-running analytical queries it can cut down on query time and free up CPU time.

DDL statements are not JITed, and for good reason - the vast majority of them are going to be I/O bound and not particularly interesting or useful to JIT. However, this is one exception: CREATE INDEX. Even though it does a lot of I/O (typically scanning the entire table) in my experience it is CPU-bound. Even moderately sized tables of tens or hundreds of gigabytes can take 10-30 minutes to index. And sometimes you just have to reindex everything, at once, like when you restore a backup with pg_restore. JIT compilation of the index creation code would be a huge and immediate benefit to just about every PostgreSQL user. And it would be a benefit: the PGCon 2017 presentation “JIT-Compiling SQL Queries in PostgreSQL Using LLVM“ which first explored JIT compilation found up to a 19% improvement in index creation time.

Reduce the memory usage of prepared queries

Like many databases, PostgreSQL has a PREPARE statement that takes an entire query with placeholder values and stores its query plan for future re-use. You can later invoke the stored query with EXECUTE, passing values in for the placeholders, and take advantage of all the precomputed query planning work. If your query is executed more than once, even if only a few times, this winds up being a measurable win. The database feature has been around for years and years and years, but over on the Python driver side, we’re finally starting to see database drivers automatically preparing and managing prepared queries for you (see psycopg3 and asyncpg). The rollout of these drivers is slow, even if they are production-ready, but over time we are only going to see more and more inadvertent users of prepared statements.

On the server side, prepared statements are per connection. They don’t take up a ton of memory, especially on the sort of beefy machine you’d use as a database host, but they take up more than you might expect them to do, to the point where it made sense for me to deallocate prepared plans when I knew they wouldn’t be used anymore. The per connection scope means that there’s no way to share query plans across connections. My first feature I’d like to see around PREPARE is a way to share prepared statements in some sort of global cache. A typical OLTP database is going to have a handful of long-lived connections, behind some sort of bouncer, and some number of hot queries, whether they are determined automatically by a driver or explicitly by the application developer. A global cache could easily be hydrated when a long-lived connection starts up or as queries are issued and easily gets some performance wins.

This issue was reported in 2019 by Daniel Migowski who provides some statistics. He happens to be working with some decently large queries which turn out to have 30-45 MB stored query plans, totalling 7 GB of RAM over his dataset of around 250 queries. It can get ugly! Under the hood, the PREPARE statement stores everything in struct CachedPlanSource, which includes the original query’s text, the parse tree for the query, the query tree for the query, and possibly the plan tree if the database decides to re-use a generic plan for each execution. All of this is potentially needed to execute a prepared statement in the future.

But I’d also like to draw your attention to an email by Tom Lane later in the thread where he suggests some techniques to reduce the memory usage of the individual query plan. In particular, one of the suggestions is:

A totally different idea is to make a variant version of copyObject that is intended to produce a compact form of a node tree, and does not create a separate palloc allocation for each node but just packs them as tightly as it can in larger palloc chunks. [...] The stumbling block here is that nobody is gonna tolerate maintaining two versions of copyfuncs.c, so you'd have to find a way [...] to autogenerate the copy functions from annotated struct definitions; people have muttered about that for years but not done anything.

Some background on PostgreSQL’s memory management circus: there are a lot of tree data structures in the backend of the database. Originally, when the database was written in Lisp, the trees and their nodes were what I’d broadly call object-oriented giving them standard interfaces, constructors and destructors. When porting to C they created this copyObject function that can traverse trees, inspect each node and do the appropriate work to copy it. I’d still call it object-oriented but it is definitely C-flavored OO. This high-level interface over trees is the first of three legs of memory management in PostgreSQL.

The second leg is memory contexts. Each context has a malloc-like interface (palloc, as mentioned by Tom before) that allocates within a memory context. All memory contexts are organized into a hierarchy of contexts, and there is always a current context used by palloc() that gets set by the backend as it moves through the different stages of query execution. By scoping allocations with the same lifetime you reduce memory fragmentation but also make cleanup easier - when the user is finished with a query or it aborts for whatever error, you can tear down the relevant parent memory context and have all of its children recursively cleaned up.

The memory context API is also object-oriented in that it is a standard interface implemented by several sub-allocators. They make up third leg of memory management in the database. The most frequently used one is AllocSet, an arena allocator. This is the actual code that calls malloc() and free() to get whole free pages from the system. AllocSet maintains its own freelist to avoid too much round-tripping between PostgreSQL and the system allocator and groups smaller allocations of the same size onto the same page to reduce fragmentation.

There’s a lot of flexibility in this system but there’s also a decent amount of overhead as each step is allocating one or more structs for its own bookkeeping data. Avoiding this overhead is what Tom is proposing: there could be a special copyObject that figures out the full size of the tree it has to copy, makes a single allocation, and packs everything into it, avoiding most of the overhead from all the small data structures needed for the prepared query. This is the second feature I’d like to see attempted to save PREPARE memory usage.

An entirely new allocator?

AllocSet is a mature battle-tested, and pretty simple piece of code which has sat at the heart of PostgreSQL since 1999 with little change to its design since then. There is something to be said for the longevity of such a fundamental piece of code and I would not be surprised if any alternate implementations can’t beat its performance just because of its simplicity. I’d like to see someone try to slot one of the modern malloc implementations (e.g. jemalloc, mimalloc, tcmalloc) into PostgreSQL. Even if it isn’t a win I’d like to see the results. Modern implementations take advantage of everything we’ve learned about writing fast memory allocation on modern hardware and operating systems and have seen lots of real-world testing. I also don’t think the palloc()/memory context API needs to change at all, modern allocators typically have an arena allocator interface and could be slotted in to replace the memory context API without API breakage. Some concerns would be legal licensing or performance losses from synchronization (the PostgreSQL backend is single-threaded and thus any effort spent on locking or synchronization is wasted) but I am hopeful that there is a malloc that can fit the bill.

Batching query plans for joins

In the terminology of the PostgreSQL query planner, when you join two tables, the database picks one table to be the outer relation and one to be the inner relation. The outer relation is the table that gets scanned first and the matching rows it emits are checked one at a time against the inner relation to get the final, joined result row(s).

Most uses of a foreign data wrapper have you talking to another database system over a network. With that comes overhead from using the network and some per-query overhead in the foreign database. It would be bad to repeatedly query the foreign database in an inner loop as you'd pay this overhead each time. Unfortunately, if the database picks your foreign data wrapper to be the inner relation in a join you get one query for each row emitted by the upper relation and pay that overhead each time.

What I'd like to see is an execution technique (in PostgreSQL terminology, a path or upper relation) that is capable of batching rows from the outer relation, sending them as one query to the inner, and sorting out the results. I have a hunch this would be a win for typical FDW use cases but I have not tested it. This would be useful for the built-in PostgreSQL-to-PostgreSQL FDW (fdw_postgres) which has to emit one query per outer relation row but once implemented the same path could be used in other wrappers. It would be a useful addition to the core database but it's actually possible to implement your own paths in an extension via the little-known custom scan path API.

gil Sat, 01/28/2023 - 12:51pm