Taylor Swift s/t - Amazon Linux 2

Taylor's self-titled release came out in 2006. As an unknown talent she had to spend several years prior practicing her songwriting and slowly putting together what would be a chart-topping debut album. Everyone's familiar with it, most enjoy it, but it's rarely someone's favorite. Amazon Linux 2 is a fork of RHEL, a distribution with its own lengthy history, and is the culmination of effort in and outside of Amazon. Nearly everyone's worked with Amazon Linux at some point and sure, it gets the job done, but nobody really reaches for Amazon Linux outside of AWS or can say it's their favorite.

Fearless - Maemo

Taylor hit the typical sophomore slump with Fearless. It was time to bring something new to the table and she just didn't quite pull it off. The result was forgettable, and so is Maemo, Nokia's attempt to break the iPhone/Android duopoly. Did you know that after the Maemo/Moblin merge an open source community kept maintaining the original Maemo codebase? Nobody does because nobody used this stuff. It was a desperate attempt to stay relevant and a cash grab.

Fearless (Taylor's Version) - Tizen

Speaking of desperate cash grabs, Taylor Swift re-recorded and released Fearless to turn her one-time album purchases into ARR (Taylor's Version). Tizen was spun up from the ashes of MeeGo (the child of Maemo and Moblin) by Intel and Samsung. Intel wanted to get in on the smartphone revolution and Samsung wanted a knife in their back pocket should Google threaten their independence but in the end nobody really got anything out of it.

Speak Now - NixOS

Taylor's third release demanded creativity and our girl delivered on Speak Now with the start of her turn from country to straight pop music. NixOS also struck out for new territory. They've invented creative ways to package software and deliver on an immutable, declarative and transactional system but the usability of the system and the overgrown DSL used to package it is still an open question.

Speak Now (Taylor's Version) - Guix

Nobody requested a re-release of Speak Now but Taylor did it anyway. Nobody asked to wrap Nix with Guile Scheme and use it to build an entire distribution but someone at GNU did it anyway.

Red - Arch Linux

No, it's not Red Hat, even though she's literally wearing a red hat. Red is the fan's album, it doesn't introduce anything new but instead carves out a comfortable, intimate space for her listeners. The Swifties love it, fans often rank it as their favorite, but it has little weight out in the greater world. Arch Linux has passionate fans all over the web but at the end of the day it's all but missing from the server room.

Red (Taylor's Version) - Manjaro

It's a fork of Arch. Nobody asked for either of these and yet here we are.

1989 - Android

Taylor finished her pivot to pop music with 1989 and put out her most popular and accessible album yet. User friendliness and accessibility is a must if you want to capture the broadest audiences. Google created a Linux with working sound out of the box, a useful sandboxing model for user applications and a coherent user experience but only after throwing out the entire Linux userspace. As a reward for their efforts Android is the world's most popular software platform and Linux's world domination is complete.

Reputation - Ubuntu

Blaccent Taylor is an embarrassment and simply shouldn't exist. Ads in /etc/motd are an embarrassment and Ubuntu shouldn't exist. Bonus: both embraced bold, unconventional color schemes.

Lover - Red Hat Enterprise Linux

Lover is another pop album, doing a little bit of everything and has broad appeal. It turned into a sleeper hit years after its release when Cruel Summer started charting again in 2023. RHEL's been around forever, it's used for everything, it has its fans and it simply isn't going away.

Folklore - Slackware

Folklore is something new and stylistically unlike any of her previously releases but also her attempt at finding a nostalgia that could resonate with her now diverse fanbase. Slackware's an OG, older than everything on this list, but even back then it was nostalgic for the commercial UNIX workstations of the 80s.

Evermore - Gentoo

Evermore came quickly on the heels of Folklore and kept its theme, writing about the timeless characters of the past and the way things used to be. However it wore out its welcome and wasn't nearly as well-received as Folklore was. Gentoo had its moment in the sun in the 2000s but you don't seem to hear much about it anymore. The nostalgia for building everything from source, via shell scripts, on the dinky one and two core machines of yesteryear may have inspired ebuild but we've moved on.

Midnights - Debian

Taylor Swift focused on what really matters in Midnights: accessible pop music, TikTok-friendly hooks (nice!) and simple, reliable operating systems found on every server and in every container across the globe.

1989 (Taylor's Version) - ???

Who knows? Maybe it'll be your favorite distro, maybe it'll be one of those Android vendor kernels that hacked in massive security vulnerabilities to ship some questionable junkware feature.

Banning or regulating TikTok has been discussed for years but now Congress may actually make a move towards it with the RESTRICT Act. Lawfare has the history of TikTok regulations and a brief summary of the RESTRICT Act and gives some useful analysis, although I wish they'd write more on it given the popular attention the bill's had so far. Vox also has a summary without a lot of analysis. I'd like to talk through my analysis of the issue and give some of my perspective.

The Story So Far

TikTok is different in important and interesting ways. Taking a step back in American history, and looking at the bigger picture of American media, we've always had media that is, at a fundamental level, at odds with the state. You don't sell newspapers with happy headlines. But at the same time, also at a fundamental level, all American media has also been aligned with the state's interests. This is a tension but not a contradiction. The state wants to propagate itself and preserve its influence. The media wants the same for itself, but it is also reliant on the state as they are part of a shared American society. Should the state's existence or influence be threatened the media's existence or influence is also threatened. Both the state and the media are, at some level, in alignment because they are both comprised of Americans sharing an American society and all the popular media we've seen so far in American history has shared this alignment.

There is also an overlooked view of the relationship of the state to its subjects, one that most often arises between the state and corporations. The primary relationship between the state its subjects is the one where the state gives direction, in the form of law, and enforces the law. It is the relationship where the state is superior to its subjects. In the case of the media, the First Amendment puts strong restrictions on the state's power in this relationship by restricting the laws the state can pass. But, in a case that is often overlooked, the relationship between the state and one of its subjects can also be one of equals. This typically looks like a business deal or any place where the state might sign a contract. In those situations the state and its subject are working together: both have their own agency, interests and goals and both are negotiating to find a mutually beneficial outcome. This is a form of soft power that the state can wield to accomplish its goals. Some people are uncomfortable with this soft power, they think the only way the state should ever exert power is in the explicit style of the first kind of relationship. But this soft power is inevitable and those who would wish it gone are naive. It arises because the the state and its subjects are not one and the same, a distinction that comes from liberalism itself and does not exist in authoritarian states. The state, enforcing a liberal order, has distinguished itself from its subjects, giving them agency, and this separation creates the opportunity for soft power when they happen to interact as equals.

The Issue At Hand

Those are some useful ideas but also abstract ones. They could apply to TikTok but they also apply to any number of apps, mass media platforms, or specialist products like SolarWinds. However, the topic at hand is much more concrete: how could TikTok be used to further the objectives of the Chinese state? Let's walk through some examples of the levers available to them through TikTok.

First, the most extreme and most unlikely thing China could do with TikTok is to push zero days to everyone's phones. This is unlikely because it is an act of war, and if it did ever happen we'd have more important things to worry about. Cybersecurity is still important, we should promote security practices in the personal, private and public spheres to protect us from realistic cybersecurity threats. But even in the context of a hot war a digital Pearl Harbor is not necessarily a great idea. It is most beneficial for China if the general population's view is "what did China ever do to us?" and not "those bastards hacked everyone's phones and screwed everything up!" This is a case where soft power can be more useful than hard power, an important idea I will come back to later.

Next is the collection of user data. This includes the data mediated by the phone's security model (the permissions you've granted the app) but also your own interactions with the app. For TikTok, that would include your interests (based on the videos you've watched and interacted with) and any social graph you've built in the app (any phone contacts you've imported, any friends you've added on the platform, any DMs you've sent to friends). That is a lot of very personal information! I am sympathetic to people who want to take a broad brush across the industry to reduce the total amount of information any app is allowed to collect on a user or reduce the legal uses of that information. If the current baseline of restrictions is unacceptable, let's raise that baseline.

Even if we find the baseline is acceptable the consumers of that data are different actors with different goals. The technical and legal baseline is just a baseline and it is not unreasonable to have different, more stringent standards for different users of the same data. Facebook and Google collect this data because they want to sell you ads, and TikTok does as well. But the Chinese state has its own interests and should they find TikTok or its data useful to promote them they have TikTok at their disposal to do so. TikTok is different, in ways we haven't seen before in American history, and it should not surprise us if it is subjected to a higher legal standard. We can't reason about it in the exact same way we'd reason about the American media and it is not unfair or illiberal to treat it different than the way we've treated the American media.

Getting to specifics, what is some of this soft power that China could exercise through TikTok in its role as part of the media? Remember that in 2020, a video of the execution of George Floyd went viral on the internet, reaching every part of American society. We saw race riots in several large cities, but don't overlook that we also saw civil rights protests happen basically everywhere. There were civil rights protests in Vidor, Texas - a place that is arguably still a sundown town! The simple ability to promote a strategically chosen video to the American public is a soft power. It is a tricky power to use but when the stars align it can have vast influence over the entire American society. TikTok has this soft power and unlike the media of the past it is not broadly aligned with the American state. TikTok is different.

As an aside, I am not saying anything about the specifics of the George Floyd video, but instead I think it is a familiar and modern example of the strength of the soft power of viral media. In a more concrete example, TikTok could promote English-language videos of the Chinese-picked leadership in Tibet and Tibetan Buddhism, popularizing the idea of an aligned Tibet and covering up China's colonization of Tibet. You and I may know that Tibet was invaded for geopolitical reasons but it's much easier for the public to digest videos of a happy and unified Tibet over content that explains snoozefest details like the early security objectives of the People's Republic of China. And even in this example it would only be under the scope of the RESTRICT Act if it was done to steer American decision-making on a topic.

This is also an opportunity for the state-and-media-as-equals soft power introduced earlier. Fundamentally, the First Amendment prevents the state from telling the media what to do. Coincidentally, if the colonization of Tibet results in unhappy headlines, the media might print it anyway. When the state and media interact as equals, the state now has an opportunity to bring attention to stories not covered by the media. But it could also bring forward its intelligence or analysis on what the media has or hasn't covered and the media, still acting with its own agency, can run or bury stories in its own response to what it's learned or negotiated.

The question at hand is not one of liberalism or to the degree that our government should set a baseline of liberalism for the media at large. Liberalism is what created this soft power and got us into this situation in the first place! We are instead looking at the paradox of freedom (see: the large body of philosophy on this topic) and the question is if American state should be allowed to defend itself and its interests against the narrow case of specific organizations that are fundamentally unaligned with itself. I think it does, and I hope you agree with me - liberalism doesn't uphold itself, it has to be defended. That is what the RESTRICT Act sets out to do, and I think it does an OK job at it.

What does the RESTRICT Act do?

If you really want to know, go read the bill or go read the Lawfare article. But in short, the RESTRICT Act doesn't call out TikTok by name. Instead, it seems to share the same ideological framework I've laid out here: it defines a category of organizations that are fundamentally unaligned with the American state, who at times could wield soft or hard power against the United States or its citizens, and it delegates identifying the specifics of the abuse of that power and the state's response to it to the executive branch. This sort of delegation is typical and expected under our Constitution, and this new power comes with additional protections: it can only be exercised against information technology systems and it limits that further to systems under the control of foreign adversaries. The scope is narrow enough to make it consistent with my understanding of liberalism and I'd go as far as saying it's consistent with all but the most naive conceptions of liberalism.

Once you get past the bill's scoping of covered transactions the powers delegated to the executive branch are intentionally broad. The purpose here is to give the government flexibility beyond prohibition ("you can no longer do this thing") as a response, which gives the government flexibility to scope its own exercise of power against the covered transaction. I am sympathetic to criticisms that the law could add explicit scoping of delegated powers here but understand that it may be hard to effectively draft it. The already explicitly-stated prohibition is the most impactful of powers the government could exercise and it seems like a necessary power to let this bill have any effect at all. There is little point in limiting the seemingly-broad "any other action" when prohibition is going to be a necessary part of the bill anyway. However, I admit that I could just lack imagination on a better solution.

I’ve put in my fair share of time with PostgreSQL. It has no shortage of functionality and I believe its expansive feature set drove a lot of its growing popularity in the 2010s. As my time with the database has worn on I’ve come up with a handful of features that I wish the database had. Some of these are ideas borrowed from other systems, some are inventions of my own. The real tragedy is that I simply don’t have the time to implement the patches myself.

–i-am-a-dummy mode

This idea comes from MySQL whose stock CLI client supports an –i-am-a-dummy flag. When enabled, MySQL identifies any UPDATE or DELETE that lacks a WHERE clause and fails them with an error. In other words, if you intended to DELETE FROM table1 WHERE pk = 10, started typing it into your prompt, and only got as far as DELETE FROM table1 before accidentally executing the query, the query errors out instead of having every row deleted. More modern versions of MySQL also use the flag to block queries that return excessively large result sizes, require the WHERE to reference a key column, or allow a LIMIT in place of a WHERE.

Update: User felixyz on lobste.rs points out that there is a pg_safeupdate server-side extension that blocks UPDATE/DELETE without WHERE.

Update: Andrey Borodin and Nikolay Samokhvalov of Postgres TV streamed themselves implementing this behind a GUC! They sent in the patch but it was rejected by Tom Lane.

Unit test mode (random result sorting)

The order of a query’s results is famously unspecified unless you nail it down with an ORDER BY. The flexibility is useful: sometimes you need a stable order for whatever code consumes the query results and sometimes you just don’t. But there is a secret, third thing: some code comes to accidentally depend on a coincidental ordering of the results. To help flush these out it would be neat if PostgreSQL could intentionally shuffle the order of query results if a given query did not specify an ORDER BY. I imagine this as a run-time configuration field for the database that you’d turn on when running automated tests or in a testing environment. The alternative, folding every query in your application into a subquery that randomly sorts the results, is ugly and invasive, although perhaps some ORMs could implement the feature at their level.

Update: User avianlyric on Hacker News pointed out that CLUSTER can can be used to reorder results in a pinch.

Query progress in psql

This is another feature from MariaDB. It’s possible in both databases to query the database to get progress information on several types of queries, but the ALTER TABLE, CREATE INDEX, DROP INDEX and LOAD DATA INFILE queries also automatically inform the CLI client on their progress. The CLI uses this to draw a nice animated status bar with some informational messages. PostgreSQL lets you query the system catalog for the progress of several query types but has no live update functionality in the psql client.

Pandas-like join validation

I am a true database expert who has never screwed up a join before. But there are many beginners out there and it would be neat to extend the SQL language to support validation features like Pandas has for its join function. When doing a join you’d specify what relation you expect to have unique keys and which one has potential duplicates and PostgreSQL could warn or panic when this assertion is triggered. Some hypothetical syntax:

SELECT x, y
FROM t1
JOIN t2 USING (key) VALIDATE 1:m

Which would have PostgreSQL check that t1.key has no duplicate rows and errors if it comes across any. This is another place where it would be useful to have a runtime option to disable the expensive uniqueness checks.

Pandas’ “suffixes” argument, which automatically renames overlapping columns in the output, and “indicator”, which emits a new column identifying the left or right table or both as the source of an output row, are also neat ideas that seem really helpful for relational newbies trying to troubleshoot their joins. Pandas is insanely popular, it’s taught to a lot of very inexperienced developers, and I think features like these were earned with newbie blood. I suspect PostgreSQL’s users and developers don’t get that sort of exposure to newbies and their typical problems but there’s no reason why they can’t learn from the hard-earned experience of Pandas and incorporate some good ideas.

Update: User richbell on Hacker News brought up DuckDB's Friendlier SQL which is a superset of sensible extensions to SQL. I endorse all of these ideas except for the trailing comma fix - join the dark side and use leading commas!

JIT support for CREATE INDEX

Today, the JIT support in PostgreSQL is closely tied to query planning and execution. It rests upon the object-oriented nature of the execution plan and emits LLVM IR for each plan node as it traverses the plan tree. It’s an elegant technical approach, and for its sweet spot of long-running analytical queries it can cut down on query time and free up CPU time.

DDL statements are not JITed, and for good reason - the vast majority of them are going to be I/O bound and not particularly interesting or useful to JIT. However, this is one exception: CREATE INDEX. Even though it does a lot of I/O (typically scanning the entire table) in my experience it is CPU-bound. Even moderately sized tables of tens or hundreds of gigabytes can take 10-30 minutes to index. And sometimes you just have to reindex everything, at once, like when you restore a backup with pg_restore. JIT compilation of the index creation code would be a huge and immediate benefit to just about every PostgreSQL user. And it would be a benefit: the PGCon 2017 presentation “JIT-Compiling SQL Queries in PostgreSQL Using LLVM“ which first explored JIT compilation found up to a 19% improvement in index creation time.

Reduce the memory usage of prepared queries

Like many databases, PostgreSQL has a PREPARE statement that takes an entire query with placeholder values and stores its query plan for future re-use. You can later invoke the stored query with EXECUTE, passing values in for the placeholders, and take advantage of all the precomputed query planning work. If your query is executed more than once, even if only a few times, this winds up being a measurable win. The database feature has been around for years and years and years, but over on the Python driver side, we’re finally starting to see database drivers automatically preparing and managing prepared queries for you (see psycopg3 and asyncpg). The rollout of these drivers is slow, even if they are production-ready, but over time we are only going to see more and more inadvertent users of prepared statements.

On the server side, prepared statements are per connection. They don’t take up a ton of memory, especially on the sort of beefy machine you’d use as a database host, but they take up more than you might expect them to do, to the point where it made sense for me to deallocate prepared plans when I knew they wouldn’t be used anymore. The per connection scope means that there’s no way to share query plans across connections. My first feature I’d like to see around PREPARE is a way to share prepared statements in some sort of global cache. A typical OLTP database is going to have a handful of long-lived connections, behind some sort of bouncer, and some number of hot queries, whether they are determined automatically by a driver or explicitly by the application developer. A global cache could easily be hydrated when a long-lived connection starts up or as queries are issued and easily gets some performance wins.

This issue was reported in 2019 by Daniel Migowski who provides some statistics. He happens to be working with some decently large queries which turn out to have 30-45 MB stored query plans, totalling 7 GB of RAM over his dataset of around 250 queries. It can get ugly! Under the hood, the PREPARE statement stores everything in struct CachedPlanSource, which includes the original query’s text, the parse tree for the query, the query tree for the query, and possibly the plan tree if the database decides to re-use a generic plan for each execution. All of this is potentially needed to execute a prepared statement in the future.

But I’d also like to draw your attention to an email by Tom Lane later in the thread where he suggests some techniques to reduce the memory usage of the individual query plan. In particular, one of the suggestions is:

A totally different idea is to make a variant version of copyObject that is intended to produce a compact form of a node tree, and does not create a separate palloc allocation for each node but just packs them as tightly as it can in larger palloc chunks.  [...] The stumbling block here is that nobody is gonna tolerate maintaining two versions of copyfuncs.c, so you'd have to find a way [...] to autogenerate the copy functions from annotated struct definitions; people have muttered about that for years but not done anything.

Some background on PostgreSQL’s memory management circus: there are a lot of tree data structures in the backend of the database. Originally, when the database was written in Lisp, the trees and their nodes were what I’d broadly call object-oriented giving them standard interfaces, constructors and destructors. When porting to C they created this copyObject function that can traverse trees, inspect each node and do the appropriate work to copy it. I’d still call it object-oriented but it is definitely C-flavored OO. This high-level interface over trees is the first of three legs of memory management in PostgreSQL.

The second leg is memory contexts. Each context has a malloc-like interface (palloc, as mentioned by Tom before) that allocates within a memory context. All memory contexts are organized into a hierarchy of contexts, and there is always a current context used by palloc() that gets set by the backend as it moves through the different stages of query execution. By scoping allocations with the same lifetime you reduce memory fragmentation but also make cleanup easier - when the user is finished with a query or it aborts for whatever error, you can tear down the relevant parent memory context and have all of its children recursively cleaned up.

The memory context API is also object-oriented in that it is a standard interface implemented by several sub-allocators. They make up third leg of memory management in the database. The most frequently used one is AllocSet, an arena allocator. This is the actual code that calls malloc() and free() to get whole free pages from the system. AllocSet maintains its own freelist to avoid too much round-tripping between PostgreSQL and the system allocator and groups smaller allocations of the same size onto the same page to reduce fragmentation.

There’s a lot of flexibility in this system but there’s also a decent amount of overhead as each step is allocating one or more structs for its own bookkeeping data. Avoiding this overhead is what Tom is proposing: there could be a special copyObject that figures out the full size of the tree it has to copy, makes a single allocation, and packs everything into it, avoiding most of the overhead from all the small data structures needed for the prepared query. This is the second feature I’d like to see attempted to save PREPARE memory usage.

An entirely new allocator?

AllocSet is a mature battle-tested, and pretty simple piece of code which has sat at the heart of PostgreSQL since 1999 with little change to its design since then. There is something to be said for the longevity of such a fundamental piece of code and I would not be surprised if any alternate implementations can’t beat its performance just because of its simplicity. I’d like to see someone try to slot one of the modern malloc implementations (e.g. jemalloc, mimalloc, tcmalloc) into PostgreSQL. Even if it isn’t a win I’d like to see the results. Modern implementations take advantage of everything we’ve learned about writing fast memory allocation on modern hardware and operating systems and have seen lots of real-world testing. I also don’t think the palloc()/memory context API needs to change at all, modern allocators typically have an arena allocator interface and could be slotted in to replace the memory context API without API breakage. Some concerns would be legal licensing or performance losses from synchronization (the PostgreSQL backend is single-threaded and thus any effort spent on locking or synchronization is wasted) but I am hopeful that there is a malloc that can fit the bill.

Batching query plans for joins

In the terminology of the PostgreSQL query planner, when you join two tables, the database picks one table to be the outer relation and one to be the inner relation. The outer relation is the table that gets scanned first and the matching rows it emits are checked one at a time against the inner relation to get the final, joined result row(s).

Most uses of a foreign data wrapper have you talking to another database system over a network. With that comes overhead from using the network and some per-query overhead in the foreign database. It would be bad to repeatedly query the foreign database in an inner loop as you'd pay this overhead each time. Unfortunately, if the database picks your foreign data wrapper to be the inner relation in a join you get one query for each row emitted by the upper relation and pay that overhead each time.

What I'd like to see is an execution technique (in PostgreSQL terminology, a path or upper relation) that is capable of batching rows from the outer relation, sending them as one query to the inner, and sorting out the results. I have a hunch this would be a win for typical FDW use cases but I have not tested it. This would be useful for the built-in PostgreSQL-to-PostgreSQL FDW (fdw_postgres) which has to emit one query per outer relation row but once implemented the same path could be used in other wrappers. It would be a useful addition to the core database but it's actually possible to implement your own paths in an extension via the little-known custom scan path API.

Ed. note - This is the transcription of a memo written to President Kennedy that explores the CIA's bureaucracy, explains its cultural problems, and suggests organizational reforms. It is dated June 30, 1961, not long after the infamous Bay of Pigs invasion in April of 1961. The author, Arthur M. Schlesinger Jr., was a "roving reporter" who reported directly to the President. This is a candid and well-written memo. Given Schlesinger's history with the CIA and his broad access to the entire executive branch I believe he paints an accurate picture.

Ultimately, the reforms suggested by Schlesinger did not happen. There was no dismemberment of the CIA's operational responsibilities, there appears to be no subjugation of the CIA to the State Department, and the Reagan-era creation of the Intelligence Community centralized intelligence coordination under the CIA instead of a pluralistic Joint Intelligence Board.

---

Memorandum for the President

Subject: CIA Reorganization

I submit the following views as one who worked in OSS during the war and served as a periodic CIA consultant in the years since.

On balance, CIA's record has probably been very good. In the nature of clandestine operations the triumphs of an intelligence agency are unknown[1], all the public hears about (or should hear about) are its errors. But, again in the nature of the case, an agency dedicated to clandestine activity can afford damned few visible errors. The important thing to recognize today, in my judgment, is that the CIA, as at present named and constituted, has about used up its quota. Its margin for future error is practically non-existent. One more CIA debacle will shake faith considerably in US policy, at home as well as abroad. And, until CIA is visibly reorganized, it will (as in the Algerian instance) be widely blamed for developments of which it is wholly innocent.

The argument of this memorandum is that CIA's trouble can be traced to the autonomy with which the agency has been permitted to operate and that this autonomy is due to three main causes: (1) an inadequate doctrine of clandestine operations; (2) an inadequate conception of the relationship between operations and policy; (3) an inadequate conception of the relationship between operations and intelligence. The memorandum also suggests ways in which some of these problems can perhaps be alleviated.

I. CIA Autonomy

CIA conduct three main forms of secret work: clandestine intelligence collection; covert political operations; and paramilitary activities. It carries on these functions with relative autonomy. The reasons for the autonomy are historical.

When CIA began, the State Department, still thinking too much in terms of its traditional missions in foreign affairs, looked on this new venture with suspicion and renounced the opportunity to seize firm control of CIA operations. It did not, for example, try to establish any effective system of clearance for CIA activities; and some ambassadors frankly preferred not to know what CIA was up to in their countries. After 1953 the fact that the Secretary of State and the Director of Central Intelligence were brothers further confirmed CIA's independence from supervision by State Department desks.

In the meantime, CIA grew in size and power. During the fifties it began in some areas to outstrip the State Department in the quality of its personnel. Partly because CIA paid higher salaries and even more perhaps because Allen Dulles gave his people courageous protection against McCarthyite attacks, CIA was able to attract and hold a large number of able and independent-minded men. The more it added brilliant activists to its staff, the more momentum its operation developed, and the greater role it played in the initiation of policy.

By the time that State had begun to be fully aware of the problems created by an active and autonomous CIA, it had long since missed the chance to establish its own ultimate authority. While State retained a nominal supervision over covert political operations, this was to some degree offset by CIA's tendency to present a proposed operation almost as a fait accompli; State never had title to control overt clandestine intelligence collection; and it had lost ground in such areas as overt political reporting and even in the maintenance of overt diplomatic contacts.

For its part, CIA had developed a whole series of functions paralleling already existing function of the State Department, and of the Defense Department as well. Today it has its own political desks and military staffs; it has in effect its own foreign service; it has (or had) its own combat forces; it even has its own air force. Its annual budget is about __ times that of the State Department. The contemporary CIA possesses many of the characteristics of a state within a state.

II. Doctrine

Though CIA's autonomy developed for historical reasons, it has been able to endure because there is no doctrine governing our conduct of clandestine operations. The problem of doctrine for CIA is the extent to which its various clandestine mission are compatible with a free and open society.

It is idle to argue that, because the Communists can do such-and-such, we are free to do it too. Communism ie a creed nurtured in conspiracy; and the whole point of Communist social and political organization is to make conspiracy effective. If 'fighting fire with fire' means contracting the freedoms traditionally enjoyed by Americans in order to give more freedom to CIA, no one seriously wishes to do that. Yet I do not feel that we have tried rigorously to think through the limits which the maintenance of an open society places on secret activity. Until this is done, CIA'a role will not be clearly defined and understood. The problem which must be faced is: what sort of secret activity is consistent with the preservation of a free social order?

We must begin, I believe, by accepting the fact that the United States will continue to be a nation in which politicians will ask questions and make speeches, reporters will dig out stories, newspapers will publish editorials, individuals, driven by promptings of conscience, will blurt out things harmful to the state, and so on. We do not wish to change these things and could not do so without violating the essence of our society. These things make up the framework in which CIA must operate. In short, they constitute the problem; and, as General Marshall used to say, "There's no point in fighting the problem."

There follows from [unreadable], I would think, the conclusion that secret activities are permissible so long as they do not corrupt the principles and practices of our society, and that they cease to be permissible when their effect is to corrupt those principles and practices.

Each form of secret activity presents its own problem. Clandestine intelligence collection has been a traditional function of the national state. Its rules and usages are fairly well established. In the main, espionage, if conducted with discretion, should present no great problem. However, when conducted with an open checkbook and a broad mandate even espionage can begin to push against the limits of secret activity in an open society. I consider later, for example, the question of the Controlled American Source (CAS) and whether the overdoing of CAS is not beginning to harm other [unreadable] of the government.

Covert political operations present a trickier problem. Occasions arise when it is necessary to subsidize newspapers, politicians and organizations in other countries. But corruption of the political life of another nation is not a responsibility to be lightly assumed. I wonder whether CIA has not done too much of this for the joy of it. Paramilitary activities create an even more difficult problem, if only because the problems of concealment are so much more difficult.

In general, I would suggest that any secret operation whose success is dependent on the suppression of news, on lying to Congressmen and journalists and on the deception of the electorate should be undertaken only when the crisis is so considerable that the gains really seem to outweigh the disadvantages. This suggests that the size of the proposed operation may be of crucial importance. Small operations can be done with a minimum of accompanying corruption. The greater the visibility of the operation, and more its success depends on thwarting the impulses and denying the values of an open society, the riskier it becomes, and the more urgent it is that an overwhelming case be made for its necessity.

In short, when confronted with a proposed operation, we must not only ask whether the operation is technically feasible in its own terms; we must ask whether its success requires our open society to be [unreadable] serious way false to its own principles.

III. Operations and Policy

Clandestine activities should be assessed not only to the context of their relationship to an open society but also in the context of their relationship to United States foreign policy.

CIA operations have not been held effectively subordinate to U.S. foreign policy.

1. Clandestine intelligence collection is, by charter, free from State Department control. This fact exposes American foreign policy to a multitude of embarrassments when CIA is discovered recruiting agents or developing sources in a friendly country. The recent Singapore case, when CIA, without notice to the Consul General, tried to subvert a member of the Special Branch of the Singapore Police provides an instructive example. After complications of ludicrous complexity, including an attempt to subject the recruit to a lie-detector test, it turned out that the recruit had long since informed his superiors of the CIA approach and was leading CIA into a trap, which, when sprung, produced considerable embarrassment to relations between the US and the Singapore government. There have been troubles of a comparable sort in Pakistan and in Japan (where a group of Chinese Nationals were smuggled into Sasebo[?] to work in a CIA-NSA operation).

CIA has said that, in such cases, neither the Embassy nor the Department in Washington is normally informed of this type of operation. In short, no one knows how many potential problems for US foreign policy -- and how much potential friction with friendly states -- are being created at this moment by CIA clandestine intelligence operations. Surely there is an argument for permitting State to decide whether the advantage to be gained by the operation (e.g. the information derivable from an agent within the Special Branch of the Singapore Police) outweighs the risk (e.g. exasperating the local government and shaking its confidence both in our purposes and in our sense). Clandestine intelligence operations should plainly be cleared both with the Department of State (and save for exceptional instances and on agreement of the Secretary of State) with the local U.S. Ambassador.

2. Covert political operations technically require State Department clearance. In practice, however, CIA has often been able to seize the initiative in ways which reduce State's role almost to that of a rubber stamp.

This has been partly the consequence of the superior drive and activism of CIA personnel, especially as compared with the diffidence of State Department personnel. For example, when men come to CIA with the assignment of developing covert political campaigns or organizing coups or preparing for paramilitary warfare, these men naturally fall to work with ingenuity and zeal; they probably feel that they are not earning their pay unless, say, they organize as many coups as possible. The concept of 'contingency planning' has legitimatized the concrete preparation of operations still presumably in a hypothetical stage; people did not at first notice that 'contingency planning,' when carried to the stage of recruitment and training of personnel, creates a vested interest which often transforms contingencies into apparent necessities. Thus, if a group is assembled and revved up on a contingency basis, then the failure to carry the project through (it is argued) will invite the disappointment and alienation of the group; so the pressure increases to follow through on what had started as a pure speculation. This was a central factor, of course, in the Cuban decision -- the fact that disbandment of the Cuban force in Guatemala just seemed to create too many problems and embarrassments. Having entered into relations with personalities in foreign lands, CIA has sometimes seemed to feel that we must thereafter do pretty much what they want -- or else they will blow security or even go over to the Communists. This approach has made us on occasion the prisoners of our own agents.[2]

It has meant too [unreadable] State Department, often apprised of an operation only in its later stages, is under great pressure to endorse the operation as already mounted because of the alleged evil consequences [unreadable] exercising a veto. I well remember Tom Mann's remark the day the decision was made to go ahead on Cuba: "I would never have favored imitating this operation; but, since it has gone as far as it has, I do not think we can risk calling it off." Moreover, at a time when there is increasing premium on activism, State, when it questions CIA operations or initiatives, casts itself in a prissy, sissy, negative role. The advocate of clandestine activities seems 'tough' and realistic; the opponent has to invoke such intangibles as the reputation of the United States, world public opinion, "What do we say in the United Nations?", etc., and seems hopelessly idealistic, legalistic and 'soft.'

The result of CIA's initiative in covert political operations has been to create situations which have forced policy on the State Department. This was not the original idea behind CIA. As Allen Dulles wrote in his 1947 memorandum to the Senate Armed Services Committee, "The Central Intelligence Agency should have nothing to do with policy." Yet, in the years since, CIA has, in effect, 'made' policy in many parts of the world. A number of governments still in power know that they even been targets of CIA attempts at overthrow -- not a state of mind calculated to stimulate friendly feelings towards the United States. Indonesia, of course, is a prime example.

This experience suggests that the present system by which CIA notifies State of a projected covert operation is inadequate to protect US interests. There must be some means by which State can be informed of such operations at an early enough stage to affect the conception and preliminary planning of the operation. Otherwise CIA will continue to confront State with propositions having potential impact on foreign policy but at too late a point to subject that impact to reasonable control.

3. The Controlled American Source (CAS) represents a particular aspect of CIA's encroachment on policy-making functions. [The rest of the section is redacted.]

4. Paramilitary warfare, I gather, is regarded in some quarters as a purely technical matter, easily detachable from policy and therefore a proper function of the Department of Defense. Yet there is almost no CIA function more peculiarly dependent on the political context than paramilitary warfare.

There are several reasons for this. For one thing, a paramilitary operation is in its nature a large and attributable operation and thereby, as suggested above, clashes with with the presupposition of our open society. (These considerations need not apply, however, to the training of, say the South Vietnamese in guerrilla tactics or to the support of already existing guerrilla activities.) For another, the moral and political price of direct paramilitary failure is acute for us. Communists, when they stimulate paramilitary activity, are doing what the world expects from them; when we do it we appear to betray our own professed principles and therefore cannot afford to compound delinquency by defeat. Moreover, as the recent Algerian episode showed, once we convince the world that ws are committed to a paramilitary endeavor, we will be blamed for all sorts of things. And, as the recent tractors-for-prisoners episode showed, when we do send men to possible death, we cannot lightly write them off and close the books. The Communists, on the other hand, have no scruples about liquidating a losing show.

Moreover, guerrilla warfare is fought, much more than military men ordinarily realize, in an ideological context. The guerrilla succeeds when his program enlists grassroots support; and it is very difficult indeed to beat him when the countryside is with him. Van Mook told General Marshall in 1946 not to worry about the guerrillas in Indonesia; "we are sending 50,000 men out there and will clean the situation up in the three months." Marshall, who had had experience fighting guerrillas in the Philippines, replied "It won't be so easy. You will find that they will bleed you to death." Cyprus and Algeria illustrate the difficulty of overcoming guerrilla resistance through sheer weight of military force.

Serious guerrilla movements hare been defeated only three times since the end of the Second World War: in Greece, because Tito's defection closed the northern border; in the Philippines; and in Malaya. In the last two cases the guerrilla resistance ended because of the combination of political and military countermeasures. The struggle in Malaya, as Field Marshal Templer understood (and said), was for the minds and hearts of the people; it therefore involved not only paramilitary operations but a vast educational program, organization of trade unions, political reform and an offer of national independence. If the guerrilla's power lies in his revolutionary program, the answer lies In part in meeting the needs which enable the guerrilla to rouse the countryside. "Without a political goal," wrote Mao Tse-tung, "guerrilla warfare must fail, as it must if its political objectives do not coincide with the aspirations of the people and if their sympathy, cooperation and assistance cannot be gained." He added, "Do we want the support of the masses? If we do, we must go among the masses; arouse them to activity; concern ourselves with their [unreadable] and woe."

For these reasons, paramilitary warfare cannot be considered as primarily a military weapon. It is primarily a political weapon and must therefore be subjected to close and careful political oversight. It probably should be retained in a reconstituted CIA rather than transferred to Defense. 

5. How to Establish Policy Control? Here I wonder whether the British experience might not be of value. The notable feature of the British Intelligence system is the determination to keep clandestine activity under strict Foreign Office control. This control is achieved in a number of ways: 

a) Secret Intelligence Service (MI-6) itself operates under the direction of the Joint Intelligence Committee, which has a Foreign Office chairman (until recently Sir Patrick Dean, who is now the British representative at the UN) and which includes the Service Intelligence directors and representatives of the Colonial Office and the Commonwealth Relations Office as as the Chief of [unreadable].

b) basic political warfare directives are originated, not by SIS, but by the Information Research Department in the Foreign Office, often in consultation with an interdepartmental Working Group on International Communist Fronts, and under the ultimate control of the Superintending Under-Secretary of the Permanent Under-Secretary's Department in the Foreign Office.

c) SIS covert political action campaigns must not only conform to Foreign Office directives but must be cleared with the appropriate Foreign Office geographical desks.

d) working groups under IRD chairmanship govern operations in special areas, such as, for example, Sino-Soviet relations or the World Youth Festival. 

e) a Foreign Office Staff Liaison Officer sits next to the SIS Chief, and Foreign Office officials serve tours of duty in SIS sections.

All these devices might be adapted for use by the State Department. Organizationally, this would mean that the intelligence agency would retain operational autonomy but that its operations would be at all points subject to State Department clearance -- a clearance to enforced by directive, by State Department chairmanship of working groups, and by the infiltration of State Department personnel into the intelligence agency.

Obviously this set-up would not succeed, however, unless the State Department itself were prepared to overcome its inbred habits of diffusion, negativism and delay and to take a firm and purposeful grip on the situation.

IV. Operations and intelligence

The relationship between operations and intelligence raises particularly perplexing questions. 'Intelligence' includes two separable activities: clandestine collection; and research, analysis and evaluation. The first deals with that small portion of raw intelligence procured by secret means (Mr. Dulles estimated in 1947 that "a proper analysis of the intelligence obtainable by ... overt, normal and aboveboard means would supply us with over 80 percent, I should estimate, of the information required for the guidance of our national policy"); the second involves the collation and interpretation of all forms of intelligence, however obtained and the production of estimates.

Under the British system, clandestine collection is entrusted to the Secret Intelligence Service. The research, analysis and estimating function is located in the Foreign Office Research Department.

Under the American system, CIA has responsibility for both clandestine collection and research and analysis. The State Department's Bureau of Intelligence and Research also has research and analysis responsibilities; but, in practice, CIA has established control over the machinery for producing national estimates in such a way as to reduce State's contribution to submitting its view to CIA for acceptance or rejection. In other words, where in Great Britain the Foreign Office plays the coordinating role in the intelligence field, in the United States that role has been assumed by CIA.

The argument against incorporating the research and estimate function in State was made by Dulles in 1947: "For the proper judging of the situation in any foreign country it is important that information should be processed by an agency whose duty it is to weigh facts, and to draw conclusions from those facts, without having either the facts or the conclusions warped by the inevitable and even proper prejudices of the men whose duty it is to determine policy and who, having once determined a policy, are too likely to be blind to any facts which might tend to prove the policy to be faulty."

Precisely the came argument can be used with equal effect against the incorporation of the research and estimate function in CIA -- i.e., if intelligence is too closely connected with operations, then those committed to a particular operation will tend to select out the intelligence which validates the operation.

Obviously both arguments conclude in a case for the establishment of a fairly independent research and estimate group. But, if the R & E group is too independent, one runs into the opposite danger: that is, that neither policy nor operations will be subjected to adequate intelligence checks.

The trouble with the Cuban operation, for example, was not that intelligence and operations were combined, but precisely that the Cuban operation evaded systematic intelligence judgment. The intelligence branch ([DDI?]) of CIA was never informed of the existence of the Cuban operation. The Office of National Estimates was never asked to comment on the assumption, for example, that discontent had reached the point in Cuba where a successful landing operation would provoke uprisings behind the lines and defections from the Militia. In December and February, the Office of National Estimates produced general appraisals of the Cuban situation, but these were holly independent of the Cuban operation.

I gather that, if its opinion had been invited, [DDI?] would have given quite a different estimate o t the state of opinion in Cuba from that on which the operation was based. There existed, in short, the ridiculous situation that knowledge of the Cuban operation, flaunted in Miami bars by any number of low-level agents in the operations branch of CIA, was denied to even the top officials of the intelligence branch. The Bureau of Intelligence and Research of the Department of State knew even less about the Cuban operation.

The problem, then, is to have an R & E group sufficiently independent of both policy and operations to resist the pressure to make the case for vested ideas and interests -- yet sufficiently close to be able to subject projected operations or policies to the most intense and searching scrutiny.

Where could this group be located? If the CIA were to become subordinate to State, as MI-6 is to the Foreign Office, then the R & E function might be vested in a coordinate subagency, somewhat independent of both State and CIA, yet closely connected with both in day to-day operations. The R & E subagency would receive intelligence from CIA and from State, as well as from the services and of course from public sources. It would represent, in effect, a fusion of CIA/DDI and State/ Intelligence and Research. It might also take over certain of the service functions now confided to CIA -- photographic interpretation, biographical data, foreign broadcast monitoring, overt collection, maps, etc. There might be in addition a Joint Intelligence Board with representatives from all the intelligence agencies and with a State Department or White House chairman.

V. Conclusion

The argument of this memorandum implies a fairly drastic rearrangement of our present intelligence set-up. It also implies the capacity of the State Department to assume command of the situation and to do so in an effective and purposeful way. If the State Department as at present staffed is not capable of assuming effective command, this is not, in my judgment, an argument against a rational reorganization of intelligence. It is an argument for a drastic overhaul of the State Department.

 The structure which would meet the criteria suggested in this memorandum would be as follows:
 
 1) The State Department would be granted general clearance authority over all clandestine activity. This might be effectuated on the British model by the appointment of a Deputy Undersecretary of State for Intelligence, who would act for the Secretary in these matters and who might serve as chairman of a Joint Intelligence Board.
 
 2) The Joint Intelligence Board would include representatives from all elements in the intelligence community and also from the White House.
 
 3) The operating branches of the present CIA would be reconstituted under some blameless title (the National Information Service). This new agency would be charged with responsibility for clandestine collection, for covert political operations and for paramilitary activities. It would submit projects to the Deputy Undersecretary of State for Intelligence for clearance. In general, the agency would bear somewhat the same relationship to State as the Disarmament Administration and ICA presently do.
 
 4) A second semi-independent agency would be set up, again bearing a blameless title (the Foreign Research Agency), coordinate with the operations agency. This agency would be charged with responsibility for collation and interpretation. It would include CIA/DDI, the Bureau of Intelligence and Research in State, and the various service functions now carried on by CIA (photographic interpretation, biographical files, foreign broadcast monitoring, scientific intelligence, maps, overt collection, etc.). It might well be located in the CIA building in McLean.

 Arthur Schlesinger, Jr.

---

[1] Or should be -- a gross and repeated CIA failing has been its occasional readiness to succumb to the temptations of favorable publicity. The Guatemalan and Iranian operations were almost nullified by the flood of self-congratulatory publicity which followed them (e.g., the articles by Richard and Gladys Harkness, "The Mysterious Doings of CIA," Saturday Evening Post, Oct. 30, NN. 6, 13, 1954).

[2] We become prisoners of our agents in another sense too. The Cuban episode leaves the strong impression that CIA is not able to control its own low-level operatives. While the CIA people in Washington are men of exceptionally high quality, the men attracted to field jobs are sometimes tough and even vicious people motivated by drives of their own and not necessarily in political or even moral sympathy with the purposes of the operation. Such actions as locking up the Revolutionary Council, such enterprises as Operation 40 in Miami where CIA agents reportedly trained Cubans in methods of torture, such episodes as the detention and third degree reportedly administered to Dr. Rodolfo Nodal Tarafa and 16 other democratic Cubans this February - all these suggest that things go on under CIA sponsorship which CIA in Washington is only dimly acquainted.

Ed. note: the cowards at the DBA Stackexchange deleted this post. It has been re-posted here for posterity.

How do I make our data warehouse OSHA compliant?

What are the safety techniques that we should follow in our data warehouse to guarantee OSHA compliance? The existing literature isn't very helpful. I bought yellow high visibility vests for all of my DBAs but they're refusing to put them on. - asked Apr 1, 2020 at 12:01

---

• As with most questions on this network, you are encouraged to start with the official docs and update your post with specific regulations that concern you. Your General Counsel and HR should be made aware of any updates to your safety policy as well as any merges to the master branch. – Peter Vandivier Apr 1, 2020 at 12:20
• Did you try switching the hi-vis vests to the dark mode? Many IT professionals prefer that for mental health reasons. – mustaccio Apr 1, 2020 at 12:56

---

I bought yellow high visibility vests for all of my DBAs but they're refusing to put them on.

Joint and severable liability of employer and employee for OSHA compliance is a longstanding dialogue¹. For the time being, you are limited to internal HR procedures to discipline your employees for infractions.

Assuming your management troubles are sorted though, here's a few specific items you may find helpful to consider for workplace safety in your data warehouse

• Employees should not lift more than 50 GB (48 GiB in Puerto Rico and USVI) of data unassisted. Those on reduced work orders are limited to 500 MB (450 MiB)
• High-bandwidth cabling should be labelled and insulated. Any network pipe of capacity >1GB/sec must not be serviced unless two or more technicians are present.
• Clearly labelled emergency stops should be easily accessible at all partition switching locations.

Stay safe out there.

– answered Apr 1, 2020 at 12:18 Peter Vandivier

If you've ever purchased real estate you may remember the deed: a legal document describing the property and declaring that the grantor, the old owner of the property, is giving it to the new owner, the grantee. A copy of the deed is usually filed with the county. This leaves a chain of deeds on file going all the way back to whatever was the original grant for the property. In the original thirteen colonies of the United States this chain goes all the way back to the original grants from the English monarch.

Tracey and I recently purchased a home in Mecklenburg County, North Carolina whose register of deeds maintains a searchable database of deeds filed since 1990. But there are still more deeds: the county has records going back to 1763. Incredibly, the county has scanned in all of the old deeds as well and made both databases available for free! The old deeds don't have a text search, which is unfortunate but not surprising - deeds before 1920 or so are hand written in cursive. So my mission is now clear - can I trace the title of this land all the way back to the original grants from the Lords Proprietors of Carolina?

The county organizes its deeds into books, and the deed is indexed on its page in the book. These are written out like 36356-796, where the first number is the book number and the second is the page number. Yes, there are a lot of books - they've been counting up since 1763! That deed is the start of my search as it's the deed that transferred ownership to Tracey and myself.

Deeds are pretty formulaic. They are practical instruments, meant to cleanly transfer ownership of property to the grantee. In the original thirteen colonies of the United States they often use the traditional English legal practice of metes and bounds to describe the property as if you had walked around its boundary. The most recent deeds simply reference a specific lot on an official map, also filed with the county, instead of a metes and bounds description of the property, but once we get past the immediate subdivision history of the land we'll see metes and bounds used exclusively. The metes and bounds also put the boundary line in context with the property around it - for example, when a boundary runs along a public road or highway the road will be mentioned by name. This helps with locating the property on a modern map as many of our Mecklenburg County roads have not changed names or locations in over a hundred years. However, there is still some sleuthing required when names change, or when the deed only references nearby properties by owner's name at the time of sale, such as in 3032-203:

BEGINNING at an old iron, which iron is also the common northeast corner of Mrs. R. C. Mitchell and the southeast corner of the property of A. A. Porter and J. F. Neely estate, thence with the line of A. A. Porter and J. F. Neely estate North 31-59 West 376.62 feet to an old iron; thence with the line of Duke Power Company North 22-49 East 868.34 feet to an old iron located on the line of the property of Gambrell Investment Company (formerly Marshall Reid); thence with said line of Gambrell Investment Company South 20-47 East 1,834.59 feet to an iron located on the line of Mrs. L. E. Carothers, and which iron is also located South 66-17 West 356.06 feet from an old iron; thence with the line of Mrs. L. E. Carothers South 66-17 West 627 feet to an iron; thence continuing with the line of Mrs. L. E. Carothers, the line of the 1.11-acre tract conveyed by Mrs. Helen C. Carothers (Mrs. L. E. Carothers) to Gambrell Investment Company, and the line of Mrs. R. C. Mitchell North 14-24 West for a total of 874.03 feet to the point or place of BEGINNING, containing 20.33 acres as shown on a survey prepared by E. L. Faires, North Carolina Registered Surveyor, dated September, 1968. BEING the same property conveyed to Robert O. Brown and Wife, Doris H. Brown, by deed of Nancy Porter Kirby (Widow), and Clara Sledge Woods (Widow), dated __ ___, 1967, and recorded in Book 2881, at Page 442, of the Mecklenburg Public Registry.

I know that the deed is in Steele Creek township in Mecklenburg County as it is written elsewhere on the deed. The names are also familiar - the Carothers and Neelys come up many times in Steele Creek as I've done the title research. But where is this land actually located? There's no mention of county roads, postal addresses (although deeds never mention postal addresses) or natural features like creek crossings or confluences. There is no reference to any map (although sometimes one does get filed with the county) and the survey mentioned at the end is not filed with the county. At least it mentions the prior deed and I can look up that one in hopes it has more detail. But if that can't be found you have to find the neighboring properties mentioned by owner and hope you can triangulate the location from their context.

The "irons" in the description are metal stakes left in the ground by surveyors to designate the corners of the property. Yes, this comes with all the issues of accidentally or intentionally moved or misplaced markers. As you go back in time you start seeing trees, tree stumps, or the course of creeks and rivers in these descriptions making it more and more challenging to accurately draw a map of the boundaries of the property. The property may be formally specified and records kept for hundreds of years but the passing of time makes it a challenge to recover the actual property boundaries.

In a fascinating quirk of legal history in the US the complete history of the title has to be computed and double-checked to make sure that the land can be successfully sold. Should there be any problems in the past with the ownership or sale of the land you risk having older owners, their heirs, or an entire tribe of Indians sue to regain their rightful ownership over the land. This research is done by a title insurance company before they sell you a title insurance policy which pays out should you be subject to one of these lawsuits and lose ownership of your land.

The title search

Starting with the latest deed at 36356-796 the most recent history of the land is easy to trace. Each deed mentions the previous deed by book and page number making it easy to jump to the previous deed. In the year 2004 we reach deed 17821-200 where D. R. Horton, the homebuilder, sells the newly constructed home to its initial owners. A few months before that is deed 17203-933 where Provident Development Group sells the house to D. R. Horton. I am not sure what Provident Development Group is or does but I suspect they are closely related to D. R. Horton and heavily involved in the initial construction of the Huntington Forest subdivision.

In April of 2002 the Provident Development Group acquires the land from the Steele Creek (1997) Limited Partnership in deed 13509-753. This is the first time Sarah Belk Gambrell comes up in the title history. Sarah is a partner in the the partnership and mentioned by name in the deed. We'll also find that she's a major land owner in Steele Creek. But what is her story?

If you're from Charlotte the name Belk will jump out at you. It's the same Belk as the chain of department stores (think Macy's, but southern) and the John Belk Freeway in Uptown. The department store was founded by William Henry Belk in 1888 and his son John Belk was a former mayor of Charlotte. Sarah Belk was William's daughter and passed away in 2020 at the age of 102. Sarah worked in the family business, served as an officer in the corporation, held board seats and was a major shareholder. Belk was bought out for $3 billion dollars in 2015 and Sarah had a lifetime of salary, dividends and sales from the business. Her obituary mentions the extensive philanthropic support she gave to minority interests in Charlotte. But with that money she was also heavily invested in Charlotte itself as a substantial land owner, speculating on the growth and future development of Steele Creek.

The 13509-753 deed was Sarah's sale of part of her holdings in Steele Creek to create the Huntington Forest subdivision and some other nearby properties. However, at this point the deed trail runs cold. This deed does not mention the book and page of any prior deeds like we've seen so far. Luckily, this transaction is in the post-1990 online database which allows us to search on grantor/grantee name and try and figure out what happened here. I am not sure why this deed didn't mention the previous deed but there are a couple of clues. First, there is a lawsuit between the Steele Creek Limited Partnership and the city of Charlotte to acquire land for a sewer pipe. This lawsuit ends in default judgment a year later in favor of Charlotte, awarding them title to a narrow strip of land for the sewer. There's also some deeds giving the NC Department of Transportation extra room to widen the nearby NC 160 state highway and improve the NC 160 and NC 49 intersection. The existence of the lawsuit sounds dramatic - did the city have to FIGHT to get the land from Sarah? - but my understanding is that it is a lot more mundane than it sounds. The lawsuit is simply the legal process required for the city to acquire title or an easement for this utility work. Because it is a lawsuit, run by the courts, following their due process, it gives the land owner a fair chance to defend their ownership of the land. And because the trial expired in default judgment I believe both parties were in agreement from the get-go and a lawsuit with default judgment is the simplest and cheapest way to get it squared off. Sarah Belk Gambrell is a multi-millionaire practicing sophisticated land speculation, I don't think her lawyers would let a contentious lawsuit go unanswered. However, I also haven't dug into the trial beyond the court's documents ordered to be included in the deed book.

The title search also turns up deed 5399-284, from 1986, which is an antecedent to the prior deed. This is a transaction between a different Gambrell Investment Corporation to Sarah Belk Gambrell herself. I call it the mega deed: it does not feature the metes and bounds of any property, but instead references 38 different deeds, acquired between 1961 and 1973, and lists just the book and page number of each deed. Unfortunately there is no map here or any hints - I need to go through each one and figure out if Huntington Forest is in any of them.

As I dug into this I found that Sarah owned property close to what is now the intersection of I-485 and NC-160 on both sides of the interstate and also large parts of property along NC-160 south of that intersection. She owns the Huntington Forest area but also land around the NC-160 and NC-49 intersection, including what is now the RiverGate shopping mall in its southeastern corner. It took several hours, going deed by deed to figure out which ones were clearly not contributors to Huntington Forest and which ones were possible ancestors.

The top third or so of Huntington Forest comes from deed 2881-596, the Ewart property. It is one of a few pieces of property carved out of the J. L. Milwee estate, which very helpfully had a map filed with the county.

Tract "C" is the source of the Ewart property and others.
The map also gives us some concrete hints about land ownership and location in this area. First, you can see the modern Steele Creek Road (NC-160) and Erwin Road in their current alignments, making it easy to situate this on a modern map. The McClintock Presbyterian Church lot is also visible on the map. It was founded in 1865 as the first Presbyterian church for the newly freed Black slaves in Mecklenburg County and the deed to that property dates to 1867. The J.M. Corouthers property is also a name that comes up in many of these property descriptions. And finally, there is the Knox brothers property right below Tract "C". It is clear that Tract "C" is the top part of what is now Huntington Forest because it's nestled in the intersection of Steele Creek Road and Erwin Road. But to figure out if our property is in the former Tract "C" or the former Knox brothers land will require us to more precisely map out exactly how far north or south the Knox brothers line is.

After digging through the rest of the mega deed we find the Knox brothers property at 2881-477. Hooray! In August of 1967 the brothers Malcolm and Hugh Knox sold their family farm to the Gambrells. The deed is composed of three tracts. Tract C is further west on NC-49/Tryon St./York Road (as it was called then) and borders the Pleasant Hill Presbyterian Church that is still standing today. Tracts A and B both reference the NC-160/NC-49 intersection so they are clearly very close to Huntington Forest. But which one, if either, is going to be the ancestor of Huntington Forest? In order to find that I had to plot out the metes and bounds for each tract on a real map. Luckily, there is a nifty and free Plat Plotter website to help you with this.

To use Plat Plotter you mark the initial point on a slippy map and then type in the metes and bounds from the deed in a structured way. The description for Tract A and B both start with:

BEGINNING at a point in the center line of North Carolina Highway No. 49 (York Road), said point beginning N. 60-41-24 E. 456.00 feet from the intersection of the center line of N. C. Highway No. 49 and the center line of N. C. Highway No. 160 (Steele Creek Road) [...]

The modern intersection of NC-49 and NC-160 is slightly to the east of the historical intersection. Originally the two highways intersected as straight lines but an improvement to the intersection in the 1990s had NC-160 curve to the east before curving back to return to its former path. You can see the "hump" where NC-160 curves on the modern map above and also see the course of the original straight line path. On the Plat Plotter website I just marked the original intersection by eye.

The starting point for the metes and bounds is not at that intersection, though: it's at a very precise "N. 60-41-24 E. 456.00 feet" from it. What does that mean? That's how the surveyors encode the path around the property. "N. 60-41-24 E." is the bearing, the direction for the next segment in the boundary starting from your current location. It starts with N or S, meaning north or south, then the 60-41-24 is the degrees, minutes and seconds of the angle from north or south. The E or W at the end indicates if your bearing is turning towards the east (or clockwise from north) or turning towards the west (counterclockwise from north). To summarize, to find our property corner and the starting point for the metes and bounds you'd start in the center of the (former) intersection of NC-160/NC-49, you'd face north, turn 60 degrees clockwise, and march forward 456 feet.

Once you have the starting corner selected on the map you have the enviable task of transcribing all the legs from the deed into Plat Plotter so it can trace the boundary. Tract B, not pictured, covers a large portion of the RiverGate shopping mall south of Tryon St. Tract A, pictured below, is our jackpot. It clearly covers the bottom 2/3rds of Huntington Forest, including the house that we purchased!

The parts not included in Tract A are also informative. It doesn't run all the way north to NC-160, which is what we saw on the Millwee Estate Tract "C". There's a missing piece to the west where the J. M. Corouthers land would be.  The boundary with Corouthers roughly follows the creek (Walker Branch) which would make for a natural boundary between farms. And it doesn't include a long, narrow strip of Huntington Forest along its east side, corresponding with the Tract "B" strip in the Millwee map that was also not part of the Knox brothers property. Plat Plotter also gives you the area of the property in acres and the gap between start and close of the boundary and they both came in perfectly. Mission accomplished!

There is still more title research to be done, though. The Knox brothers sold in 1967. Where did they get the land? Once again, the deed is less than helpful - it does not reference any prior deeds. The trail has gone cold. To get the search started again I turned to genealogy research. Luckily, the 1950 Census was recently released to the public. It was written entirely in longhand but the government also made an attempt to OCR the text and index it. I found the Knox family in enumeration district 60-16, called Steel Creek. There are five people in their household: Julia (65), Jessie (55), Malcolm (56), Hugh (57) and Paul (54). They are all siblings and have never married. My interest is piqued: who are these five adult children, never married, all living together on the farm?

For additional censuses and biographical information I turned to familysearch.org, a remarkably comprehensive database run by the Mormon church. I had never used it before and was blown away by the depth of data and level of integration. The Mormons have been hard at work, pulling every public data set they can find, scanning it and OCRing it, and doing a great job of linking sources to people and building family trees. The Knox family is in good shape.

Their father, Dr. John Knox, married in 1880 in Steele Creek. He was a well-known physician in Steele Creek and had eight children. Three married and moved away from Steele Creek, the remaining five stayed on the family farm. Hugh, Malcom, and Paul - the three men who stayed to work the farm - served in World War I. Julia and Jessie were homemakers for their entire lives. After Hugh and Malcolm sold the family farm to the Gambrells they appear to have moved to a nursing home in Pineville and passed away in 1978 and 1979. I am astounded at the level of detail familysearch.org has on this family. They have the original, hand-written World War I draft cards for all the children scanned in and OCRd. They have the military record file (really just a single 3x5 card) for everyone. They have everyone's death certificates scanned in and linked into their profiles.

Looking back at the registry of deeds you can paint a picture of the history of the family. Dr. John Knox purchased a small lot for his home in 1881 (shortly after marriage and right around the birth of his eldest) just north of the modern intersection of NC-160 and NC-49. His homestead is featured on a map of Mecklenburg County from 1911 where NC-160 (Wright's Ferry Road), NC-49 (Yorkville Road), the Pleasant Hill Presbyterian Church, and Walker Branch are all visible. The names of neighbors featured here often show up on the metes and bounds of Knox land painting a picture of a small community where farms are held in families for decades and generations.

In 1885 Dr. John Knox purchased land on the eastern side of NC-160, including the land that becomes our property in Huntington Forest. You can see on the 1911 map that there are no houses on the east side of NC-160. It's prime, flat farming land and was reliably used for farming for decades until the whole area was developed. When Dr. Knox passes away in 1911 we see some deeds related to inheritance and dealings among the children but they keep referencing the original, hand-written deeds from the 1880s. This continues all the way up to deed 2703-277 in 1965 where the state pays the surviving brothers for the expansion of NC-160 and NC-49.

The Gambrell deed, made only a few years later, doesn't reference prior deeds by number or metes and bounds description. After almost 90 years the patchwork history of this land was getting a bit complex and could use some simplification but the surveys themselves also desperately needed to be redone. The original 1880s surveys were probably adequate for rural Mecklenburg County at the time but were quite inaccurate. Bearings on the metes and bounds were measured in whole degrees with no minutes and seconds, no doubt due to the technology available to those surveyors. The deeds referenced long gone roads, trees and other monuments such as a "stake on the branch with Sassafras pointers" from an 1885 deed. The more accurate Gambrell surveys translated into nice diagrams with PlatPlotter but the old surveys require substantial guesswork to figure out what they're trying to describe.

My best guess is that the southern half of Huntington Forest (the eventual Tract A in the Gambrell survey) was originally purchased by Dr. John Knox in 1885 in deed 44-10 from S. A. Carothers. The starting point is not clear from the description. I believe that when the surveyors refers to a "branch" they mean the Walker Branch creek and I also think they refer to the tiny creek behind the modern day Publix as a "drain". Even if the map location or the survey is inaccurate 44-10 seems the most likely candidate given its size, neighboring properties, and lack of other Knox tracts that could plausibly cover this amount of area.

S.A. Carothers didn't own the land for very long. He purchased the farm in 1881 from the estate of A.R. Erwin in deed 44-8. Things are getting fun now: the recording of the deed is in longhand and the scribe here has pretty good handwriting. As we get further on we'll find it harder to read handwriting and the quality of the scans gets worse. Note that the 44-8 deed is recorded along with the 1885 deeds. Modern land sales record the deed immediately but it is not a requirement. A valid sale of land can be done without recording the deed - the recording only helps establish the details of the transaction in case of a future dispute. If you lacked the money to record the deed or it was too inconvenient to get to the county seat for recording you might skip out on it for a while. In this case, after the death of Arthur Randolph Erwin, his children Margaret, Violet and Sarah sold the land to S.A. Carothers.

Arthur Erwin and his family resided in Steele Creek on that farm for decades, showing up in the 1850 and 1860 censuses. But now the deed trail goes cold - when did the Erwins get the land? There is at least one other Arthur Erwin in Steele Creek at the time which complicates the search. I see some likely deeds in the index but the scan quality doesn't let me read them. Unfortunately, my search comes to an end here. Perhaps I will have time to go in person some day to read a higher quality copy of the deed book.

CHARLOTTE, N.C. - In solidarity with the people of Ukraine, the Gil's LotD, Pro Football History.com and Omaha Poop Tracker websites have implemented economic sanctions against the Russian Federation in opposition to its war of aggression. The sanctions, a firewall on all IP connections originating from Russian IP origins, went into effect the morning of February 26th.

In 2022, Gil's LotD will be responsible for some part of the estimated $875 billion dollars in revenue from e-commerce in the United States. The sanctions intend to deny as much as possible of that revenue from Russian government officials and government-connected businesses who support the current government's invasion of Ukraine. Sanctions implemented by Gil's LotD are widely held as responsible^{[citation needed]} for the collapse of Muammar Gaddafi's Libyan government in 2011 and the Taliban's rule over Afghanistan in 2001. Gil's LotD believes that its sanctions will pressure Vladimir Putin and his government to withdraw from Ukraine and recognize Ukraine's independence as a state.

I've had this iMac 12,1 sitting around for a few years unused. The iMac was released in 2011 with a Sandy Bridge i5, a big, beautiful monitor, 12 GB of RAM or so, and an integrated, laptop-grade Radeon 6750M graphics card. I thought it might make a good web browsing appliance but I didn't realistically see it getting much use in this household. At the same time, though, I don't play games released in the past decade. So maybe I can throw Windows 7 on it, have a stable Windows machine around for once, and have an actual gaming PC? I went for it.

Backing up the old hard drive

The old hard drive had a still working but slow spinning disk drive full of personal documents and photos from my family. I use s3ql and Google Cloud Storage for my cloud backups and wanted to do a complete backup of this drive. It was easy to get the iMac to boot off of a Debian live USB, install s3ql and its dependencies on that, and do rsync to backup the whole drive. However, if you read the patch notes, s3ql had some serious sounding data corruption bugs and I needed to fsck the backup volume and also run a special data verification tool related to the data corruption issue. Furthermore, that tool worked by fetching the entire GCS blob to just to verify its size. I am not positive why they couldn't just get the size from the GCS metadata but it may have been related to the on-the-fly compression they do to your blobs. At any rate my volume is huge and it'll take forever to download and validate all these blobs so I have to spin up a one-off VM in the cloud just to do this fsck stuff and it still costs me like $40+ and a few days of VM time to do the entire fsck. s3ql's been an OK tool but its maintainership has not been great and it's had a few of these unfortunate bugs, enough to give me pause and question whether I should be using it for backups. Stay tuned, maybe I'll have the time to write my replacement for cloud backup some day.

Replacing the hard drive

To make the machine usable I wanted to replace the old spinning disk with a modern but cheap SSD. I think the machine was originally disused because the disk was just so slow, making the machine a chore to use. I grabbed a $60 internal SSD from a big box store and went to work.

This is one of those all-in-one iMacs where they've got the entire PC behind the LCD. It looks stunning now and was revolutionary when it first came out. The front of the iMac is actually a big slab of glass/plastic held in place with strong magnets. So to start any sort of internal repair you have to start with popping this plastic off without snapping it. Online videos recommended using suction cups like you'd do with a piece of sheet glass but I was able to get it with fingernails, shims and some patience.

Once the front plastic is off you can start with unscrewing the LCD which is sitting on top of everything. And it only gets weirder from there: the LCD is attached with several tiny cords, and in order to get to the spare drive bay (they have space for a spinning disk drive and a modern SSD) you have to disassemble nearly everything, as the spare drive bay is buried underneath the CD drive and against the back of the case. So I didn't do that. Instead, after sweating bullets just getting the monitor off and disassembled, I decided to just put the SSD in the space of the main drive bay and just sort of let it dangle there because I don't have any sort of tray or insert. It'll work out OK.

This machine is a work of art, inside and out. Apple really is the master of hardware design. But damn, this was one of the trickiest things I've had to do with electronics and it had me on edge the whole time. I wouldn't fault anyone for getting a third party to do this sort of maintenance for them.

Installing Windows 7 from a flash drive

Everything is put back together and it's time to install Windows. I have an ISO, I can just clone it to the flash drive with dd and tell the Mac to boot from it, right? That worked just fine with a Debian ISO on this exact same machine. Unfortunately Microsoft wasn't as thorough with their installer design as Debian is and the Mac wouldn't boot it. At this point, things start getting ugly:

• I grab one of those free Windows VMs for testing Internet Explorer and use Rufus to create a NTFS-formatted Windows installer disk. Doesn't work.
• Apple's Boot Camp Assistant supports creating Rufus-esque boot USB drives with the correct partitioning and drivers to boot on a Mac. I do this on my daily driver Mac, running macOS 10.15, and it doesn't work, because Apple has dropped support for Windows 7.
• I create a High Sierra VM from scratch as that was the last release of MacOS X to support creating a Windows 7 boot USB drive. This does not work.
• Thinking it was something about the virtual machine USB passthrough or something I copy the Boot Camp Assistant out of the High Sierra VM and run it under Catalina to format the USB drive. This does not work.
• At this point I am desperate. I wasn't planning on putting MacOS on the new SSD, but while trying to boot all of these USB drives I notice that Apple's firmware has a "Network Recovery" option to download the MacOS X installer and boot without any operating system or installation CDs. I give that a shot, and it downloads the venerable MacOS X 10.7 Lion onto my SSD. That works fine.
• Lion ships with a Boot Camp Assistant that supports Windows 7! Hooray! However, it doesn't support creating USB installers. You have to feed it an installation DVD, it formats the disk to leave a Windows partition and puts Apple drivers on it, and then boots off of the DVD. Luckily the iMac 12,1 shipped with a DVD burner and I have a very old stack of blank DVDs somewhere so I start burning my ISO to disk.
• I have to burn two DVDs because they're old and rickety and a teeny bit scratched but eventually I get a good one. Boot Camp successfully partitions the disk and gets the Mac to boot into the Windows installer. Finally, we're making some progress!
• Windows can't figure out what Boot Camp was trying to do with Windows partitions so I have to tell it to wipe the partition Apple made for it and install Windows into an empty filesystem. So, no Apple drivers on first boot. The DVD is also slightly damaged, preventing me from installing Windows 7 Enterprise, so I have to settle for Windows 7 Ultimate. But that installs and works fine.

Installing Windows drivers for the iMac 12,1

After the last installer reboot I am greeted by the Windows 7 desktop but with absolutely no drivers - no network card, no WiFi, no video card, no sound, nothing. I am so close but still so far from a functioning machine.

Boot Camp provides a standalone download of ~500 MB worth of Windows drivers for 64-bit Windows 7. This includes Apple-proprietary drivers for parts of the hardware but also the third-party stuff like the graphics card and network cards. I kept those on a USB drive and figured it would be one quick install to get everything working, right? Well, Apple's installers really didn't like my machine, for whatever reason. The installers didn't think I was running Windows on an iMac and wouldn't even attempt to install any drivers. I figured it was some sort of chicken and egg thing: because the drivers left by Boot Camp before Windows was installed got wiped, there must be some key drivers that I missed out on that allow the other driver installers to recognize that this was a Mac. Either way, I now have to bootstrap this entire machine's drivers from scratch.

First, I load up the driver for the Ethernet card. I figure with just that one I can use Window's driver discovery and update functionality to fetch the rest of the drivers for my hardware. I forget where I got this driver - it may have been shipped with Windows, it may have been pulled out of the Boot Camp package above. Boot Camp ships a top-level installer that was giving me errors, but that installer calls several sub-installers to handle each driver. Many of those sub-installers were also giving errors but I believe the Ethernet driver ran fine when ran independently of everything else and brought the card up right away.

Windows Update

Now that I have internet access I kick off Windows Update to download the inevitable patches but also hoping to get all of the drivers I need. However, Windows Update simply doesn't work. My Windows 7 ISO had SP1 slipstreamed into it, and it dates from 2012-2013 so it may even have more updates included. But at some point either a certificate expired or SSL settings changed or something and Windows 7 can't even communicate with Windows Update to get the list of available updates. Some quick Googling turns out I need a special KB update to get around this hitch. But even then I am having trouble - the old Internet Explorer lacks the certs/SSL support to view the installer page on Microsoft's website! I install modern Chrome, which of course ships with modern SSL and certificates, and Chrome will display the download page for me. But there's something about the installer download itself it doesn't like - another cert error or malware detection or something - and it won't allow you to download the installer and gives me no opportunity to override it. So I wind up having to browse to the update's web page in Chrome, copy the URL for the installer off of there, and paste it into Internet Explorer which will gladly let me download the installer. The patch works and we're back at it - Windows Update is working.

Upon reboot I am greeted with 163 urgent updates. This was the start of 3-4 hours of downloading, patching, and rebooting. There's detours in there to install some more KB updates to get around further issues with deprecated/changing security code (something about SHA2 hashes of update packages?) and fully download and install every single update it wants. The SSD is a lifesaver here but I'm still stuck on the wonderfully CPU-bound and single-threaded Windows Update.

I had hoped to be greeted by a fully functioning PC with drivers after finishing all of the updates. But that was not the case - Windows was unable to find any of the drivers for whatever reason, even when instructed to do so manually. It is now on me to find the rest of the drivers and bring them all up.

Drivers, part two

Although the Boot Camp driver installer isn't working I find a sub-installer at BootCamp\Drivers\Apple\BootCamp.msi. I believe I have to run this one from the command line to get around hardware checks. it installs most of Apple's Boot Camp drivers, especially the one-off ones provided by Apple directly, but I'm still left without a graphics card driver and WiFi driver, which are actually the only two I need to use this for gaming. Darn.

AMD still distributes old Radeon drivers for all of its graphics cards. I download a few versions but none of them are letting me install - they don't think I have a Radeon! This is troubling. Ultimately, I go to a somewhat-sketchy bootcampdrivers.com and get their patched driver which installs smoothly and brings up a working graphics card. I'm OK with leaving theirs installed as that driver is probably as modern as I'll get.

There's a Broadcom WiFi driver in the Boot Camp driver package and I am able to install it. However, it can't enable the WiFi chip once installed. I try using the generic Broadcom driver that ships with Windows (via the obscure "Install drivers on my own machine" dialog, which seems to be a relic from Windows 98 and incredibly can't be resized to show more than four of the hundreds of drivers available at a time) but this Broadcom driver does not work either. This is really unfortunate - I am incredibly close to a working system! Why won't the WiFi work, did I damage it or not reconnect it properly when putting the machine back together?

There's yet another sub-installer in the Boot Camp package that I try out, this one at BootCamp\Drivers\Apple\AppleSoftwareUpdate.msi. My hope is that this thing, once installed, will be able to detect my hardware and pull the appropriate drivers from Apple. And at first it seems promising - I think I did get some updated drivers, and I did see an installer package for WiFi support. But at the end of the day, even after installing Apple's own WiFi driver updates, I still didn't have a driver that could turn on my WiFi card!

Well, the answer turns out to be that I didn't have a Broadcom WiFi chip. After Googling the hardware identifier string I found that I had a Qualcomm Atheros AR938X. What the heck, man. I don't know if Apple even shipped drivers for this in the Boot Camp driver package. The best source for the driver online appears to be the incredibly sketchy ath-drivers.eu. Hopefully none of this stuff is compromised, but if it is - hey, it's a gaming PC.

Once that is installed the WiFi hardware comes up perfectly. Windows 7 does not yet support emoji and UTF-8-encoded WiFi network names so I have to join a mojibake SSID. This does not seem to impact the performance of the driver.

Conclusion

To test the hardware out I install Overwatch which is probably the most modern game I play. It defaults to the lowest graphics settings but appears to be getting a stable 30 FPS on the practice levels. It seems like my mission is a success!

This is archived from the original post still on the Internet Archive. I was bowled over by the lack of awareness here so I just had to archive it. There are several images in the original blog post that I haven't backed up. However, I did pull down the real whopper - an excerpt from the termination letter - and put that up.

---

Preface

Hello,

I was employed by Hudson River Trading (HRT).

The company never provided due holidays to UK employees. The issue of holidays was brought up countless times by me and other employees, during the time I worked there. All requests were systematically ignored.

The problem could have been confirmed in 5 minutes of legal review. Looking at the employment contract (28 days) and looking at the holiday allowance in the absence tool, Namely (25 days).

The company simply explained to any employee raising a question about holidays, that they are not entitled to holidays and they’re wrong. When the UK office grew and the issue escalated further, because there is genuinely a problem that remained unaddressed (employees were risking losing their job or their bonus for missing days of work they don’t comprehend), Hudson River Trading responded by firing employees who questioned holidays.

I was fired for requesting holidays for UK employees. Upon termination, the company unilaterally refused to pay notice period, refused to pay non-compete and refused to pay outstanding holidays. I had to bring legal actions against the company, who largely ignored the requests from my lawyers and left me hanging with legal fees, the company required me to sign a very strict settlement agreement in order to get a reference letter and waiver of non-compete, or otherwise to see my career my livelihood and my family shattered. It was made abundantly clear that anybody who reports any concern will meet the same fate as me.

Hudson River Trading has zero regards for the law, their employment contracts, or financial regulations. I can’t comprehend the length they are willing to go, retaliating and silencing employees, just to prevent UK employees from having a few days of holidays. It’s absurd.

This article is a protected disclosure to highlight exceptional serious failures at Hudson River Trading.

I am going to cover my experience with Hudson River Trading in great details. This is a very long read, about 20 pages of text and 20 pages of legal documents. Hold on.

TL;DR

1) Let it be known that Hudson River Trading never granted due holidays to UK employees, the company only granted 25 days off (later 26), ignoring both employment law and their employment contracts which entitled employees to 28 days. The issue has been raised countless times and Hudson River Trading has resorted to terminate and silence employees who spoke up.

2) Let it be known Hudson River Trading does not pay notice period, does not pay non-compete clause, and does not pay outstanding holidays upon termination.

3) Let it be known that Roberta Yuan (HR and legal) and Prashant Lal (partners) are terminating and retaliating against employees who report concerns (aka. protected disclosures).

4) I am formally reporting Roberta Yuan to be formally investigated for exceptional wrongdoings and embezzlement. She runs both HR and legal in the company (extreme conflict of interest). She is using her position and drawing company funds to drive retaliations against employees and cover her own wrongdoings.

5) Prashant Lal is formally signing the papers as the director, which makes him personally responsible and liable for everything. Other than signing, his involvement is unclear.

6) HR and legal functions are compromised. I formally warn employees to NOT raise complaints/grievances, at the risk of their job and their personal safety. In fact the wiki page to raise HR questions is actually giving the lawyer as the point of contact.

7) I encourage current and past employees to come forward and raise concerns and documents directly to the FCA. The FCA can be reached by filling the contact form online https://www.fca.org.uk/contact. Documents and attachments can be provided by email directly to whistle (at) fca.org.uk with Case Number #53063 Hudson River Trading in the object.

Life at Hudson River Trading

First Contact

The first day I was off was 26th December, known as Boxing Day, a bank holiday in the UK.

The 27th of December I was awakened at 5am by HR (or whom I thought to be HR), to check where I was because I did not show up to work the previous day.

From their words, my manager was worried why I wasn’t in office and whether I was alright. He escalated to HR to contact me and check

.

The 26th of December was a bank holiday in the UK. The company was closed. It was formally written in the company policy that the company is closed on that day, see wiki page on Holidays. Oh boy I did not want to face disciplinary actions for being off on a day the company is closed. U_u

Notice that 5 AM London time is midnight in New York time. The employee had to stay all night just to wait to contact me?

Holidays Absence or Absence of Holidays

The next bank holiday was around April. https://www.gov.uk/bank-holidays#england-and-wales

Same thing happened. I was off and the company was closed. Only to come the next morning and face my manager telling me I missed a day of work and he reported me. What?!

I had to explain again to my manager that it’s a bank holiday in the UK. It’s not a work day here. The same pattern repeated many times.

My manager insisted to formally take a day off if I am off, under HR instructions. I had to explain that doesn’t follow my employment contract. I wouldn’t be able to take holidays either way because I don’t have enough days off in Namely, the absence tracking tool. The manager referred to HR to sort out the problem. HR explained that I do not have holidays I thought I had. Well, what they stated did not match my employment contract.

The company only granted 25 days off per year to UK employees in Namely (the absence request tool). However UK employees should have 28 days off in addition to days the company is closed as per their employment contract.

Statutory Rights

Under British law, full time employees are entitled to 28 days off a year, including the bank holidays. (It’s possible to have 8 days other than the 8 bank holidays; the total has to be at least 28 days).

Contractual Rights

Most companies give more than the statutory minimum; it is set in the employment contract. It’s common in London to have between 25-30 days, in addition to bank holidays.

This is page 3 of my employment contract. I’ve asked other employees and they all had the same clause.

The contract means that UK employees are entitled to have 28 days off in addition to days the company is not open for business (typically bank holidays for UK businesses). It’s not clearly defined.

For reference, in British employment law, a “Working day” is formally defined: For the purposes of the Companies Act 2006, in relation to a company, a day that is not a Saturday or Sunday, Christmas Day, Good Friday or any day that is a bank holiday under the Banking and Financial Dealings Act 1971 in the part of the UK where the company. (For reference, there’s always 8 bank holidays a year and they are fall on Monday to Friday). https://uk.practicallaw.thomsonreuters.com/3-381-0053

The firm had its own definition, that was shifting and unclear.

Holidays Absence or Absence of Holidays

I lost count how many times I found myself confronted to Human Resources and/or my manager for planning to be absent or for being absent on a closed day.

Hudson River Trading only granted 25 days off and expected employees to deduct bank holidays from that.

• Under British employment law, employees are entitled to 28 days off per year (counting in the 8 bank holidays). Employees are given less than the statutory minimum (25 minus some bank holidays). It’s a breach of employment law.
• Under employment contract, employees are entitled to 28 days off per year + days the office is closed. Employees are given less than 28. It’s a breach of contract.

I very specifically raised the request to Human Resources to review their UK employment contracts and correct the holiday entitlement to 28 days, multiple times, for myself and other UK employees. The request was always ignored. Sometimes they’d say they’d look into it and come back, but there was no follow up.

This was a general issue affecting ALL UK employees of Hudson River Trading. There had been countless complaints reported and it’s never been actioned. The issue had been ongoing for years I’ve been told, before I worked for the company.

In US English, this was ground for class action lawsuit against the company from every UK based employee against Hudson River Trading… if only class actions were a thing in the UK.

Unilateral Deduction

At some point end of April I noticed an automated email from the absence tracking tool Namely, a request had been submitted on my behalf to deduct a day of. Then much later a second email, the request had been approved.

I wasn’t informed that a day has been deducted except for the automated notification. I do not know who took the initiative to do that or who approved or why.

This was reducing the holiday entitlement further away from the statutory minimum, to a point where employees won’t have enough days even when taking most bank holidays off (which the company expect you to work). Shouldn’t do that.

Contractual Failure

Hudson River Trading doesn’t follow the contract or the law. What are they doing then?

One of the root causes of the holidays issue was the employment contract (or at least that’s what I thought at the time). A normal employment contract should state how many days are off, as in “The employee is entitled to X days off per year in addition to bank holidays” (most tech companies in London have X between 25 and 30).

The contract says employees are entitled 28 days off in addition to “days the company is not opened for business”. What are those? Nobody knows. The contract of employment for Hudson River Trading is unclear. It’s a notable mistake on the part of whoever drew and reviewed the contract.

The company tried to define closed days in a wiki, the wiki was unclear and it kept being modified (which is unlawful, it’s a contract and it cannot be modified unilaterally like a wiki).

Employees should have had 28 days off, the company only granted 25 days off. The company wanted to deduct (some) days the office is closed; however they have to be in addition as per the employment contract.

I raised these issues specifically I don’t know how many times. I’ve asked HR to formally review the contract with a lawyer. It should take 5 minutes to look at the contract and the Namely allowance, to realize that employees have never been granted due holidays. The lawyer should be able to work out what closed days is supposed to mean (and flag the lack of clarity as a problem).

One time I brought the issue of holidays to HR, she replied that she can adjust the policy, a wiki, to delete the extra holidays we’re not supposed to have (the company really doesn’t want employees to have days off). I had to explain that the company cannot delete holidays unilaterally. Employment terms form a contract in the UK and they cannot be edited unilaterally. This applies to holidays and most internal policies, they’re considered to be part of the employment contract in the UK.

What To Do?

The same pattern repeated over and over.

– Before or after a holiday, my manager would tell me I shouldn’t be off.

– I’d have to explain that the day is off because it’s written in my employment contract. And I can’t take the day off even if he wanted me to because I don’t have enough holidays allowance in Namely.

– My managed would report me to HR to sort out the problem of holidays, to who I would repeat the issue and ask for the holiday allowance and/or the contract to be corrected.

– HR would systematically dismiss the request and say there’s no holiday. Though once in a while, they’d say they’d look into it and come back to me. They never did!

– I was bleeding to death from the fear of losing half of my compensation or my job, at any minute.

April-May-June was supposed to be a chill period covered with bank holidays in the UK. Instead every single week was problematic and a source of unwarranted disciplinary actions for employees. Nobody understands what days employees are supposed to work or not.

Employee might be fired over this, to no fault of their own. Employees might face half of their compensation cut off suddenly. Hudson River Trading is a financial company with a large part of the compensation paid as discretionary bonus, confrontation is not how to increase bonus. Your mileage may vary, whether you try to take any day off and whether your manager or HR decide you cannot have that day. It’s quite inconsistent of course because nobody understands holidays.

There were new joiners in my team in April. They were supposed to start on a bank holidays. Had to explain that there won’t be people in the UK to welcome them because it’s a holidays, got directed to HR to sort out the problem, who insisted there are no holidays and we’re expected to work. New joiners thought they would have another day instead and they will have other bank holidays, well, nope. Expect a shock for employees coming from other companies and losing 5-10 days off compared to their previous role. Had to warn new joiners and bring them up to speed slowly about the company. The guy before me disappeared under very questionable circumstances. I might be next at this rate. Any of us might be next.

The general advice for developers in bad situation is to immediately leave. It’s almost became a meme on the internet, just quit. Well, it’s not that simple, notably because of claw-back clauses and non-compete.

Claw Back Clauses

I’ve looked at my options to leave the company. It is not possible to leave the company.

The employment contract has claw back clauses for 12 months, that’s pretty long. To leave the company an employee must repay their sign-on bonus if there was one (I had one) and part of their compensation. Assuming the clauses are enforceable, vary with circumstances.

I estimated that if I were to leave the company, either willingly or unwillingly, I’d have to (re)pay the company about £50k. The amount is increasing each month.

(Well, I estimated wrong but I only realized that months later, the claw back only applies to 4.1.b about the sign-on bonus, it doesn’t cover the discretionary compensation in 4.1.c, I missed the small (b) on the first reads. That’s really unfortunate for me. )

The company might try to claim a lot of money back in the event that they terminate employees. It was very scary. It made the problem of holidays much worse, waiting to explode in the face of the company, there is no easy way to leave willingly or unwillingly.

See contract section 4.2 for claw back and section 14.

Non-Competition Clause

Of course to leave, one must first get a job. It’s really difficult to get an interview, let alone a job. Financial companies are very few and very selective.

The abuse of non-compete clauses is rampant in the industry to prevent employees from moving. Other companies will simply not interview or cancel later stages because of non-compete. Non-compete clauses are an enormous problem to change job in the UK and they are considered very enforceable.

The employment contract with Hudson River Trading had a stringent non-compete for 6 months.

Between the non-compete and the claw-back clauses, it’s really difficult to leave Hudson River Trading. Prospective employees should be warned that when they join Hudson River Trading, there’s no leaving.

I thought, if something went horribly wrong (and things are not going great because of the holidays problem) the company will at least have to pay the non-compete for 6 months and I won’t be left to starve. Let it be known that Hudson River Trading unilaterally cancelled their non-compete.

Haemorrhaging Employees

I found out that the way my body reacts to extreme stress is to bleed. My nose just starts bleeding anytime for seemingly no reason. It’s stress!

It came to a point where I was bleeding 3 or 4 times a day, every day. This was my body warning me of life-threatening conditions. I’ve never been confronted to stress so extreme in my life than when I was working at Hudson River Trading. The worst is that there’s no way out between the claw back clauses and non-compete.

I had to turn off my webcam during most meetings because of the bleeding. If other employees noticed I stopped showing myself, I’m sorry, you should know that it’s because I was dying, I had to cover the blood.

Maybe I should have taken a doctor’s note and get off-sick, but how to get time off-sick when the problem that makes you sick is that you cannot get time off???

HR Announcement

After months of raising grievances, HR finally decided to take action to resolve the issue, by unilaterally reducing holidays. Wait, what?

Notice the first line, confirming that multiple employees have been raising issues for a while.

This email modified the holiday entitlement from 25 days off to 26 days off and made Boxing Day (26th December) a working day. (It might not be clear but that’s what happened after this email).

It effectively removed one day off (employee should have 28 days entitlement + closed days like Boxing days).

There is a wiki page, for the official company policy, specifying 26th as a day off, the wiki was edited out to remove the day. It’s unlawful, the company cannot unilaterally deduct and change holidays. This is part of employment contract and cannot be edited unilaterally. (Not that you could get away easily with the 26th anyway, I was off the last 26^th December and was awakened the next day at 5 AM for not showing up to the office).

That announcement did not help. It was still unclear what days we are supposed to work or not this year. It did not match the employment contract. It did not grant statutory holidays. There were many more bank holidays coming up in May and June and it’s not clear when employees are supposed to work.

I can’t comprehend how did this announcement ever passed 5 minutes of legal review? It’s unlawful. It doesn’t address the problem.

Had to go back to HR once again to report all the points above and try to clarify holidays. The HR lady looked legitimately surprised that she cannot unilaterally cut holidays like that and said she would get back to me. There was no follow up.

Escalation

Nothing happened despite more grievances. I could not go on like that not knowing when to work and constantly being under afraid, there was a genuine major issue, that was leading to get employees fired or bonus cut or lawsuit or worse.

I escalated the issue to the partners with UK employees in copy.

I received a reply from Jason, who’d said he’d personally look into the matter. No idea who Jason was but he seemed important. (Remember the name, it will come up later).

At about the same time, I was contacted by another individual to get me into a meeting, this was lead HR and my manager’s manager informing me I am suspended until they investigate the matter. I insisted again that the issue is real and if they took 5 minutes to show the employment contract and the allowance to any lawyer or the lawful, the company would be immediately told that they forgot to grant holidays to UK employees.

My laptop was remotely shutdown and wiped right after the meeting. (They told they’d cut access while investigating).

Was that really necessary to review a contract?

Explosion

I was invited to join a phone call few days later, with lead HR (or whom I believed to be the lead HR) and my manager’s manager. Couldn’t even use my work laptop to join the meeting because the laptop had been murdered.

Oh boy I was naive to think that they would look into the matter.

Lead HR told me that I am fired effective immediately.

(☉_☉) That’s legally not possible, it is unlawful to terminate employee after making a protected disclosure, they can’t terminate me now.

HR informed me, precisely, that I am not being fired, the company is simply breaking my probation period, because I am still in probation period and I failed for poor performance.

(☉_☉) I am in utter shock and silent. I am NOT in probationary period or under a PIP. There is no issue with my performance. In fact my last conversation with my manager was that I am doing great and I will see my bonus increasing this quarter. The company is just making up reasons that do not exist.

HR repeated that I am terminated for performance issue. The manager added that I should never have sent this email to the partners, this is extremely inappropriate to escalate the issue like that and he can’t keep me on the team.

(☉_☉) Silence. Notice how HR is trying to make up non-existent issues about performance while the manager is formally telling me that I am being fired for making a protected disclosure. The reason for the termination is confirmed. I am now officially a whistle-blower. (That sucks, I don’t want to be a whistle-blower, what I wanted was my laptop unlocked, holiday allowance corrected to 28 days, and get back to work)

HR explained that the company is not paying the non-compete, the company is paying me 1 week to cover the notice period, but as an exceptional gesture the company is generously giving me 1 month of pay as severance to cover the troubles and the time to find a new job.

(☉_☉) There is no word for how shocked I am. The company cannot unilaterally decide to cancel the non-compete like that. I always thought if something went horribly wrong I would get the 6 months of non-compete as severance as a last resort. Nope. Hudson River Trading is unilaterally cancelling it. The notice period is actually 1 month. The company is refusing to pay the notice period and offering me to shut up for not paying my notice period. These people are insane. And they really think 1 month is enough to find a job?

I asked if they looked into holidays and if they will adjust the holidays for UK employees? and reviewed the employment contracts as requested?

HR responded that I am wrong and they will do nothing about holidays. Employees do not need more holidays.

(☉_☉) It’s extremely clear that the company hasn’t investigated anything. My access was terminated specifically to prevent me from contacting other employees and my laptop was wiped remotely in a deliberate attempt to destroy any evidence I may have had.

I insisted it’s not possible. If they had shown the employment contract to any lawyer in the UK and the holiday entitlement in namely (25 days before announcement or 26 after) and the employment contract (28 days expected), it would be pointed out immediately that the company has not granted due holidays to any UK employee.

HR insisted again that I am wrong about everything, HR added that non-disclosure clauses from my employment contract are in effect and if I talk about this matter anywhere to anyone, this is a breach of my confidentiality clause and she will pursue me on behalf of the company.

(☉_☉) Holy shit. She’s lying straight to me face about my rights and she’s plainly threatening to personally pursue me. For reference, I was just fired in retaliation for making a protected disclosure for the benefit of all UK employees, under British law all confidentiality clauses are waived and I am permitted to make a public disclosure. HR should be aware of that. (The lawyer would have pointed that too if they consulted one about holidays!) https://www.legislation.gov.uk/ukpga/1996/18/part/IVA

I asked who is the legal contact for the company? I require the legal contact in order to bring legal action against the company. I am getting a lawyer, the company will be contacted by my lawyer.

HR replied herself. The HR person who fires employees is also the legal contact. (Odd?)

I received my formal termination letter shortly afterwards.

A Personal Note

There’s something that’s really difficult to capture in text. It how problems were constantly created out of thin air and escalated out of control at Hudson River Trading (and not just about holidays).

• The first time you’re awakened at 5 am for being absent on a bank holidays. It’s painful, but it’s accidental, right? US folks don’t pay attention to UK holidays or time zones.
• The third time you’re confronted by your manager or HR, it’s not accident, there’s really a problem. I checked with other UK employees and UK employees really cannot take holidays. I’ve worked for international companies; it’s a mess when they open new offices abroad, lots of little things to sort out. The issue here is not out of the ordinary, just need to adjust some holiday’s numbers and contracts.
• Then suddenly you’re fired on the spot, your access is terminated on the spot and your laptop is remotely wiped in a deliberate attempt to destroy any evidence. The company refuses to pay your notice period and openly threatens you.

For me, there was sadly never an option to ignore the problem. My manager is a developer turned manager who is possibly the most pedantic person I’ve ever worked with. If I was (thinking of) missing any day or doubting any policy, it was systematically reported, like a bug. He actually wanted me to address the issue for other employees and new joiners of the team, which is fair, however it cannot be done if the company is not willing. It was a death march. It was bound to end up with the issue resolved or the team eliminated.

Lawyer Up

Time to get the lawyers involved. Not like there was a choice. The company refused to pay the notice period or the non-compete or to offer any severance.

ACAS

First thing was to call ACAS and explain to get advice. https://www.acas.org.uk/contact

They replied the basics:

• What you made was a protected disclosure.
• The company terminated you following the protected disclosure.
• Under British law, most protections for full time employees do not apply until 2 years of service, however if you are terminated for making a protected disclosure about a statutory right, which you were, protections apply regardless of time of service and you can sue for unfair dismissal.
• There is no cap to the level of compensation (we’re talking UK, not US, the compensation is peanuts anyway)
• You must get a lawyer ASAP.
• There’s only 3 months to bring a claim to the employment tribunal, starting from the moment you were terminated.

On Normal Abnormal Termination

Let’s cover how a normal “abnormal” termination was supposed to happen. When the company terminates an employee, the company should offer a severance package and a settlement agreement.

• reference letter of employment
• non-competition pay AND/OR formal non-competition waiver
• holidays pay
• notice pay
• severance pay
• payslips (separately later)

Note: I ought to include payslips and tax forms last. They don’t go into the settlement agreement but better make sure that they are provided to employees soon enough and that the amounts due are paid. Without the document above, including payslips and letter of reference and non-compete, there are major troubles to move on.

Hudson River Trading didn’t do any single item out of that. Not a single one. In fact, from their perspective they simply broke my probation period (I wasn’t on probation).

The settlement agreement goes to layers for revision. The employee and the employer each gets a lawyer. You have to find a lawyer, who will get back to the company and negotiate the terms. Law firms usually offer a standard package to negotiate the terms of a settlement agreement.

In the UK, there is a time limit of 3 months from the date of termination to sue the company in employment tribunal. It’s really short. The employee has to decide what to do, quick, the timer is running.

Without anything on the table, I had to pay a lawyer thousands of pounds just to start the process. I had to pay the full legal fees myself because Hudson River Trading did not do their job. And they will refuse to cover legal fees that they have incurred on me.

Termination Letter

Here comes the termination letter.

The first page stated what was said during the call.

• The company is breaking my probation period; I was not in probation period.
• The company is giving me 1 week of pay to cover the 1 week notice; the notice period is actually 1 month.
• The company is terminating me for poor performance and claiming I am under a PIP; there is no issue with my performance and there was no PIP.
• The company is not paying the non-compete clause, unilaterally, and did not provide a formal waiver of non-compete to forward to prospective employers.
• The company is firing me for making a protected disclosure; yes, it’s explained in writing

What’s interesting is the page 2, see below. It acknowledges in writing that there had been concerns reported about holidays for months, which remained unresolved. I resorted to making protected disclosure to partners and UK employees hoping to get the issue addressed. That made some individuals unable to cover their wrongdoing and they elected to terminate me.

This page alone should be enough to prove wrongdoing and retaliation from Hudson River Trading. That page alone should be enough for the company to terminate anybody who initiated that. If the document is escalated to regulators like FCA or SEC, there may be grounds to start an investigation and distribute fines.

The termination letter is signed by Prashant Lal, one of the partners. See page below. I’ve seen him few minutes during one induction for new joiners and seen him giving end of quarter presentations. Gotta admit, he did not strike me as a guy who would move heaven and earth to fire employees for few days of holidays.

Did Prashant actually read and sign this document for real? If yes, was Prashant explained the nature of this document, specifically meant to retaliate and silence employees making a legitimate protected disclosure. Was he explained that by signing this unlawful document he is personally liable and exposed to actions before courts and regulators?

On HR Announcement

I complained many times that the company doesn’t grant holidays and the HR announcement was unlawful.

Letter from my lawyer to the company, regarding the announcement and following termination:

Behold! The company doesn’t grant holidays and the HR announcement was unlawful.

That was determined in the first hour of meeting my lawyer, in the first minutes really, after being provided the employment contract and HR announcement.

Why was the company unable to figure that out in spite of getting recurring complaints about holidays for months?

On Holidays

What holidays are employees supposed to get? I’ve been trying to determine that for a long time.

As of 1^st February 2021, the answer was 28 days in addition to the following days which were the closed days defined in a wiki:

• Friday January 1^st 2021, New Year Day
• Friday April 2^th 2021, Good Friday
• Monday April 5^th 2021, Easter Monday
• Monday December 27^th 2021, Christmas Day

(It got pretty hairy if one wants to look at the history, not that it matters. Employees were never granted 28 days off. The wiki specifying these days was edited to remove Christmas Day, which was unlawful. There was another wiki for the previous year, which was deleted, and showed something else.)

On Non-Competition

The non-competition is an enormous problem. It was practically impossible for me to leave the company willingly when I was employed, because of the non-compete and claw back clauses. (Not that I wanted to leave, I actually liked my work, I wanted to stay and keep my coworkers, which is exactly why I was trying to get holidays sorted out.)

It was extremely challenging to apply for positions after termination because of the non-competition clauses.

I tried to get some interviews with financial companies, which is incredibly difficult with how few and how selective companies are these days (note that the UK is a fraction of NYC finance job pool). They systematically asked if I had a non-compete and how long? They made a whole round of HR interviews just about that! It was horrifying.

I was falling apart and crying inside in interviews. I DON’T KNOW. I don’t know if I have a non-compete, what’s the status of the non-compete when the company illegally terminated me and refused to pay the non-compete and I am pursing legal action against the company to get it? Nobody knows. (Of course you can’t say that in an interview).

It was inhuman how much Hudson River Trading and their non-compete and their catastrophic handling of contractual matters is destroying my life and my career. And they did not bother to provide a formal waiver of non-compete when they decided to waive it unilaterally so I can’t show prospective employers that I do not have a non-compete. If a prospective employer asks to check my employment contract, which is a possibility for trading companies, they will see that I can’t work for them due to non-compete.

I wish I could say it’s unlawful to unilaterally cancel but it’s not that simple. There’s little general rule about non-compete in the UK, non-compete cases are decided on a case-by-case basis. Typically the non-compete will be considered by the court when deciding compensation.

As an employee my only recourse was to formally sue Hudson River Trading in court to enforce the non-compete against the company. It’s horrifying how non-compete are one-sided in the UK.

And there was a lot of money and details behind the non-compete, 6 month of salary, making it very important. Ideally this requires a lawyer who understands non-compete (who will say that it is ruled on a case-by-case-basis) as well as protected disclosures as well as financial regulations as well as general employment.

About Performance

The company claimed there were performance issue with me, it is false.

• In my last weekly meeting with my manager on a Wednesday, my manager informed I was doing great and my bonus would increase this quarter.
• On the following Monday I was terminated by HR for issues with my performance. It’s non-sense. There are zero issues with my performance.

The company claimed that I was under a performance improvement plan. It’s false. I was not.

There is something though that I ought to cover though. My very first week-on call there was an alert and an outage of the compute grid. Due to a bug that’s always been present, it just happened to trigger the first week I was on-call and took about an hour to resolve. The following Monday my manager asked me to join a video call. Had to wait a little while because I was in a meeting, can’t be everywhere. Unbeknownst to me, this was HR and my manager waiting, to inform me that I am being put on a performance improvement plan, following poor handling of the on-call issue, then HR immediately left because she had other events to attend to. (They couldn’t bother to arrange a time where people are free and make sure every is aware they need to be present?).

An email was sent to me afterward. I formally replied to contest placing new employees on a PIP on their first week on-call and asked for training and asked to change manager rather than getting fired.

I talked to manager and HR separately the next day. I was deadly afraid to be fired. They each separately assured me they are not planning to fire anyone, it’s a very normal procedure followed with numerous employees and they rarely fired people. (Understand why I am bleeding to my own death now? The combination of the company constantly pursuing people with no reason in combination with holidays issue and claw back clauses and non compete clauses, it’s horrifying and there’s no way out)

There was never any follow up of any sort to that event. It was a one-time rant. (HR actually did not realize performance improvement plan had a specific meaning in the UK, this isolated event wasn’t one. https://landaulaw.co.uk/performance-at-work/ )

I talked to some coworkers about it. They warned me that this sort of circus happened before. The person before me disappeared under questionable circumstances; it was all extremely worrying for everyone. My role when there will be new joiners, soon, will be to warn them and ensure that they do not get canned by accident, just like other people warned me.

When Hudson River Trading started firing UK employees to cover up the holiday issue, Hudson River Trading tried to justify my termination by claiming that I was under a PIP for repeated poor performance and claimed that automatically extended my probation period. Well. It’s false and it’s unlawful. There is no performance issue. There is no PIP (that absurd rant that was never followed up isn’t one). The probation cannot be extended unilaterally without ever informing employees, if it can be extended at all.

Payslips and GDPR

There was an enormous problem after being terminated. I didn’t have payslips. (The company paid, it just did not send payslips). I was left unable to get any payslips.

There was a system to download payslips from, I tried to access it but I couldn’t access any of my payslip anymore. Access was terminated as part of the company shutting down all access?

I had to send a GDPR request to Hudson River Trading to request information, screenshot below. The company has 30 days to process the request.

The first line about HR profile was supposed to include payroll and tax forms, though that may not be clear. Communications are going through lawyers; it’s formal, slow and costing a fortune for each message.

Behold! The company simply ignored the GDPR request. No reply. Nothing at all.

It was an enormous problem because I need these documents to ask for any missing payment and to sue Hudson River Trading on various grounds. There is a 90 days delay to start legal action against the company and that required figuring missing payment and having these documents as soon as possible. The company not processing GDPR request timely or at all, was depriving me of the ability to seek legal recourse and to report wrongdoing to regulators. Also, it’s a breach of GDPR regulations.

The request got nowhere at all. Hudson River Trading will never provide anything because they know that all their communications and documents are damning evidence of illegal activities. Hudson River Trading was probably playing the watch too, because I might just miss the deadline for the employment tribunal if they stall forever.

The only thing the request achieved was the company openly threatening me. The company put a line into the later settlement agreement (which I had no choice but to sign) to retroactively cancel my GDPR request and prohibit me from sending any more GDPR request. If I send any request for any information of any sort, I will be immediately punished and requested to pay the company £35000 in damages (the sum referred in clause 3.2 is £35 000). See screenshot below, section 7.6 and 7.7.a.

They company also tried to waive my rights to make a subject access request or similar. See clauses 6.1 waiver of claim (see the settlement later in this article).

Was that legal? Can the company refuse to process GDPR requests and retaliate with punitive damages on the requester? Can the company waive rights to make GDPR requests?

There was supposed to be a data protection officer in the company, don’t know who it is, were they informed about the request and the retaliation?

The GDPR being fairly new, there is little information about what to do or the risks. I filled a report to the ICO, don’t see a case number. Sadly they are overwhelmed and only a tiny fraction of requests are processed. It’s unlikely it will ever be processed. https://ico.org.uk/global/contact-us/ 

Holiday Pay

I eventually managed to payslips after a while. That was difficult.

The company paid the notice period in full, after the notice from my lawyer that they are required by law to pay the notice. The company DID NOT pay outstanding holidays. There are thousands of pounds missing one way or another.

That took months to find out (took forever to get the payslips and the payment) and it’s seriously approaching the deadline for the tribunal. It’s bad.

For the avoidance of doubt, there may be debate about what holidays are due because holidays are a problem, that’s not the issue I am talking about there. It’s possible to count a few days more or less to calculate holidays pay. No matter the scenario, there are thousands of pounds missing. The company really did NOT paid outstanding holidays.

Had to send back a request through my lawyer for the company to pay outstanding holidays, there was a few backs and forth with the company lawyer to establish the amount. Then the company simply declined to pay, stating they “do not agree that holiday pay should be calculated in the manner that your client [me] suggested”.

What?!

It was not “my position” on calculating holidays. This was a legal request sent by my lawyer that’s costing me about £400 per email and that’s going to the lawyer of the company for review that’s costing I don’t know how much to the company. It took quite a bit of back and forth to establish the breakdown for the very specific circumstances here and with the text of laws given as reference.

The company is represented by counsel from Shearman and Sterling, a well-known silver circle law firm, that’s expensive and known to represent high profile financial firms.

Well, Hudson River Trading did not believe in their legal representations or in their obligations to pay outstanding holidays. (It’s consistent; the company did not believe in paying notice period or non-compete either!)

Seriously, are there actual people on the end of Hudson River Trading to process legal requests?

Pensions

Yet another potential problem: Pensions.

Companies in the UK must provide a pension scheme and on board employees after they join.

With Hudson River Trading, I had some forms to fill and a zoom call with HR to cover benefits and pension, around the time I joined. Hudson River Trading made pensions opt-in and they discouraged employees to opt-in to the pension. That was an interesting call to say the least (good luck proving that in court!).

Took me a little while to find and get access to the pension:

• Hudson River Trading paid the statutory amount, about £291 a month.
• It took 115 days between the moment I joined the company and enrolment email from the pension.
• That’s about 3 months of no pension contributions. No back pay.

There were legitimate questions about pensions. The company ought to make sure that employees are enrolled automatically and timely and NOT discourage employees to enroll. I think there is a legal delay of 90 days to on board or as soon as requested. How many employees missed the registration to the pension?

I wanted to bring this up formally but I was never able to, a few hours of lawyer is more expensive than the pension. At this stage, Hudson River Trading had made it pretty clear that they refuse to pay anything or refund legal fees they incurred on me.

It’s a good illustration of the catastrophic problem when dealing with Hudson River Trading. Every single thing done by the company is questionable when not outright unlawful. It’s reached a ridiculous point where it’s impossible to seek remediation because there’s too much to cover. Any employee who finds themselves in troubles will end up financially bankrupt in an endless list of claims, assuming one could get some lawyer(s) in the first place. It’s difficult to find legal representation, lawyers are specialized in a narrow field of law and the company is touching ten different things in a very complex case.

Final Settlement

Didn’t get anywhere reasonable. The company refused to pay outstanding holidays, refused to compensate for the non-compete, refused to refund legal fees they incurred upon me; the company plainly ignored the GDPR request.

The company simply considered that they’ve done nothing wrong; the company is in total denial about the existence of the law or the existence of contractual obligations or the existence of regulations.

That was two months in. There was a legal delay of 3 months to sue the company in employment court. Don’t know whether Hudson River Trading was playing the watch willingly or accidentally but that’s where things were. There is no money to cover going to courts and the company has made it abundantly clear that they will sabotage the process. I need a reference letter and waiver of non-compete NOW because it’s being an enormous problem to interview.

I was presented with a settlement agreement that gave me nothing. It was pages over pages of putting extreme liabilities over me in order to get a reference letter and a formal waiver of non-compete. I don’t think I’ve ever seen a contract in my life that was so one-sided. I can’t believe how they are shifting all the liabilities upon me, to cover their wrongdoings.

I cried for days.

I’ve tried to get interviews with some companies. I’ve had interview processes starting with a formal half hour phone call with HR to talk about the role and very specifically inquire about any non-compete obligations I may have. I want to cry. I will not be able to get a new job, not without the reference and waiver of non-compete. Background checks in finance go back 10 years; I will have to go back to Hudson River Trading to beg for a reference over the next 10 years, or risk watching my career burst in flame and go starving. The situation is catastrophic.

Sign or die.

Note: I consider that this contract was signed under duress and unenforceable. It was specifically meant to cover up wrongdoings from specific individuals, who are actively threatening me and using this contract as a mean to silence me.

See the settlement agreement.

Settlement

Recognize the extreme waivers, followed by extremely strict NDA followed by extreme retaliation clauses all backed with extreme warranties and retaliation clauses. If I break any of these conditions, I will have to pay the company £35,000 immediately upon request, at any point in my life, or other worse punishment. Of course the conditions are impossible to satisfy.

See clauses 6.1 waiver that specifically named Roberta Yuan and Prashant Lal as off limits. Somehow the individuals involved got themselves off limits, they cannot be pursued by any means, if I intend any actions of any nature against them, including going to court or regulators, and they will personally pursue me and request me to pay damages. It’s waiving right to make a GDPR request too. I do not agree with any of that at all. I do not believe that this is lawful either.

See clause 8 about confidentiality obligations. It’s void and only intended to scare me (and it really does because the company has no regard for the law). Under British law, all confidentiality clauses are waived when wrongdoing is involved.

See clause 7 about warranties. An enormous list of warranties that cannot possibly be provided but if I don’t bend over and give willingly consent to be raped, the company will destroy my career and my livelihood.

It’s inhuman. I disagree with all these pages. I was so afraid when I received the documents from Hudson River Trading. I was asking them to offer some compensation and provide reference letter and waiver of non-compete, without which I cannot work again. The document they provided turned out to be pages over pages of legalese to kill me and my family whenever they wish to in the future. No human being should ever have to sign anything like this.

FCA violations

For reference, there were a few back and forth with the company. The corrections I managed to get from Hudson River Trading was to get a 10 year time limit on the non-disclosure agreement with an exclusion for “immediate family” (which doesn’t matter because confidentiality obligations are void) and remove some warranties clauses, screenshot below (which doesn’t matter because it’s void too but I was not aware of that).

Below, please find the original warranty section that was sent to me and I negotiated to have removed. I was scared to death because it’s impossible to satisfy. (Remember for example that the company does not honor their obligations toward pensions which I am unable to inform them because they refuse to cover legal fees required to inform them). I had no idea it was null and in violation of financial regulations.

See FCA handbook section 18.5.3 about making settlement agreements. “Firms must not request that workers enter into warranties which require them to disclose to the firm that: (a) they have made a protected disclosure; or (b) they know of no information which could form the basis of a protected disclosure.” https://www.handbook.fca.org.uk/handbook/SYSC/18/5.html

Post Settlement

There was one final email from Hudson River Trading on or about the time to conclude the settlement. They wanted to confirm that I have formally handed over and destroyed any [incriminating] documents I may have.

That was plainly asking for destruction of evidence which is required by courts and regulators. It was unlawful and I could not reasonably reply or confirm that request. It was in breach of FCA handbook for preventing employees from making a protected disclosure. It was potentially a federal crime to ask for deletion of evidence if the evidence covered a federal crime (didn’t know at the time but embezzlement and fraud are very much on the table).

I should have kept the email to be able to publish it, but I couldn’t. I cannot look at it. I and my partner just cried. Couldn’t sleep. Had to delete it. It’s inhuman was the company is doing.

Hudson River is simultaneously ordering me to destroy critical evidence and building a paper trail for (not) doing it to pursue me later. I am trapped either way. The company asked me to be consent-in-writing to be raped otherwise the settlement is off the table and they destroy my career and take away my roof (no reference letter and no waiver of non-compete); either way they reserve the right to pursue me and take away anything I have at any future moment of their choosing. 😦

The email went hand in hand with Clause 9.

Prashant Lal

The settlement agreement is signed by Prashant Lal, one of the partners. See page below.

Did Prashant actually read and sign this document for real? (The digital signature from DocuSign confirms yes)

If yes, was Prashant explained the nature of this document? Was he explained that by signing this unlawful document he is personally liable and exposed to actions before courts and regulators?

The legal communications (letters from my lawyers and GDPR requests) were exclusively sent to Roberta Yuan (or to the law firm) who terminated me and self-elevated to the only point of contact for the company. No idea if anybody else has a copy of any documents.

(I note that I willingly refrained from trying to contact employees because the company is retaliating against anybody who gets involved and I do not want to endanger other employees, it is not safe!).

There was supposed to be a financial regulator in the company representing the FCA and SEC, I think a lady from northern Europe, was she informed and provided with a copy of the documents above? Forgot the name and did not have any contact information.

I’ve many times wanted to contact Prashant to talk to him personally, trying to deescalate the situation, maybe try to reduce legal fees and get a more positive outcome for both sides. Maybe confirm and warn him about some stuff, there is a real possibility the signatures were forged or that he was unwittingly talked into signing these documents. Does he comprehend the liability this personally exposes him to? Would a director sign any document like that for no benefit to them? Would a director willingly sign a document that exposes the company to enormous fines and counter action from regulators?

Maybe the ultimately irony is that this agreement formally prohibits me to contact Prashant Lal or anyone. The only recourse left open is to make a public disclosure to the general public.

The company was very successful in isolated me and preventing any contact with employees. The sudden destruction of the work laptop and the sudden termination while pretending to investigate the matter were terribly effective in isolating me and destroying evidences.

Fear and Wonder

It’s really puzzling the length the company is willing to, just to avoid giving a few holidays that employees are entitled to. It was not an accident. The company is fundamentally opposed to granting holidays. And it is committed and willing to destroy life over it. What happened to me will happen to the next person.

What will happen next when Hudson River Trading will extend to other locations in Singapore, Tokyo, continental Europe… Hudson River Trading will fire and threaten all employees there for raising any questions about local laws or customs?

It’s absurd.

Part IV: Revelations

There was a news article published in Bloomberg some time later, that highlighted critical new information that I wasn’t aware of. https://www.bloomberg.com/news/articles/2021-06-24/prop-trader-hudson-river-reaps-1-billion-in-frenzied-quarter

For reference, the article is right and the company is making over a billion dollar per quarter with about 500 employees. That should give you an idea of budget available to cover up and retaliation (it’s a shame the company forgot to put few pounds of budget aside to cover notice period and non-compete!).

Current and future employees of Hudson River should be afraid.

That should also give an idea how much money is on the table, personally for partners and high level employees on the other side, or employees who are committing embezzlement. A 7 or low 8 figures sum is peanuts for the company. People do get killed daily for much less than that.

Ownership

The first new information that came up: Jason Caroll is the owner and founder of the company. I did not know that!

There are three partners in Hudson River Trading: Prashant Lal, Jason Caroll, and Oaz Nir.

• Oaz Nir stopped actively running the company years ago, as far as I am aware. Never saw him.
• Prashant came much later; he’s giving quarterly presentations and signed all the unlawful documents to retaliate against me.
• No idea what was the involvement of Jason. Never saw him.

When I escalated the issue to the partners and UK employees, there was a global reply from a Jason but with a different surname. He said that he would personally look into that. No idea who that guy was until now, Jason is the owner and founder of the company. Obviously there was zero follow up on his part, employees got fired and threatened, and the holiday issue was never addressed.

I’ve many times wanted to talk to “whoever” runs Hudson River Trading. It’s ridiculous how every single thing the company has done is blatantly wrong and illegal. Surely the owner would not approve of that if he was aware.

@Jason: Your employees are stealing from you to threaten other employees and cover up their wrongdoings.

Embezzlement

In Hudson River Trading, there were only two Human Resource employees L. and R. These are the only two employees to raise employment questions or concerns. (Note there are recruiters who do outreach and go to university job fairs, these are not Human Resources per se). The first part of the disclosure is mostly with L., the second part is mostly with R.

This is formally explained in a wiki page. If there is any kind soul left at Hudson River Trading, please find and take a full screenshot of the wiki page to raise HR question (whole screen including URL bar) and send to the FCA whistle (at) fca.org.uk with subject “#53063 Hudson River Trading”.

Just found out after the Bloomberg article. R is not just Human Resources. She is a qualified lawyer in the New York bar. She is running both Human Resources AND is the lawyer of the company AND is the person executing all contracts on behalf of the company. That’s a lot of responsibility for a single individual, all with zero oversight. She never stated she was a lawyer and was representing the company. I always thought she was the manager/lead for Human Resources.

I believe this is an extreme form of conflict of interest. Everything is run by the same person. What would regulators think about that?

If employees read and follow the procedure to contact Human Resources, they may be contacting the lawyer instead, who may be preparing their termination. Is it normal for the lawyer of the company to be the contact for Human Resources?

Since R is not Human Resources, that means there is ONE SINGLE Human Resource employee, L, who is all alone. Is it normal for a company of 500+ employees to have only a single Human Resource person?

Is it reasonable for a regulated financial company with $4B in cash and a two digits percent of all equities trades in the US market, to have a human resource department consisting of a single new graduate? What would regulators think about that?

The entire time I thought I just needed to reach anybody who can order a contract review to correct holidays. There is nobody in the company who can process any of this. That makes sense now. There’s one single person controlling everything to cover her wrongdoing. She had me terminated and had my laptop erased the second I raised a concern she couldn’t bury. She personally threatened me for legal action if I were to disclose anything about what happened because that would highlight her wrongdoing and put her job in jeopardy.

See the elephant in the room?

This is a perfect setup to cover wrongdoing and embezzlement.

I am formally reporting Roberta Yuan, representing Human Resources and Legal of the company, to be formally investigated for embezzlement.

She’s terminated me and she personally threatened me to stay silent, or otherwise she would pursue me on behalf of the company. She personally contracted lawyers and executed contractual agreements on behalf of the company to retaliate and silence employees, her actions could not have been done on behalf of the company because it is illegal and exposes the company to enormous fines and regulatory actions, meaning it was done on her own initiative but using Hudson River Trading for the express purpose of covering her own wrongdoings. That ought to fall under the definition of embezzlement. https://en.wikipedia.org/wiki/Embezzlement

I sincerely believe that both Human Resources and Legal functions in Hudson River Trading are compromised.

It’s catastrophic. Only god knows what else may be covered up. Every contract from the company may have had similar treatment as those in this article. Any employee who report any concerns will be terminated and threatened the same way I was. Any next employee who joins Human Resources or legal will be faced with a walls of problems and risk their careers. For reference, there was a recruitment person who was fired suddenly in great fanfare earlier this year, I think she’d have to work with HR and legal closely as part of her work, what did she come across that pushed the company to instant fire?

The involvement of Prashant Lal remains unclear, except for signing all the documents and being responsible for everything. I never talked to him. (It’s extremely fishy. Why would anybody do that? There must be critical information that’s missing?).

This article is a protected disclosure for exceptional wrongdoing https://www.handbook.fca.org.uk/handbook/glossary/G3485p.html

Reporting Concerns to the FCA

I encourage all employees, both current and previous employees, to report concerns and any wrongdoings to the FCA. You can provide tips, conversations and documents to the FCA directly to whistle (at) fca.org.uk subject #53063. There is no not need to be in the UK to assist the FCA. Employees from all over the world can contribute and make the world a safer place.

No human being should be treated and threatened like Hudson River Trading has done. It’s wrong. It has gone way too far.

I tried to but I can’t live with the constant fear over my head that the company will turn around any day and try to take anything I have. I can’t live knowing that they will do the exact same procedure to me to the next person. It’s not human.

Final Disclosure

Now is December 2021, the next bank Holidays will be soon on 27^th December 2021, known as Boxing Day.

Boxing Day was a closed day, employees were not expected to work on that day and did not need to deduct one day from their allowance, it was formally written in the company policy, until the wiki was unilaterally edited to remove that day, an edit which was unlawful.

What’s the result? Are employees supposed to work or not? Who knows?

Please clarify holidays.

Please stop firing people for being off on bank holidays when no employee can comprehend what days they are required to work or not.

Zappa and Mangum^[1] are Python libraries that parse events from Amazon's API Gateway and feed them into a Python WSGI or ASGI application. This allows you to put API Gateway in front of a Lambda runtime, have Lambda execute the service requests that come in through API Gateway, and code your service logic in any Python web framework.

But what framework should you use for your next serverless REST API? I say Mangum. Both are mature products and Zappa appears to have been around longer and has more features implemented to show for it. But there are some key features, ranging from nice to have to increasingly required, that Mangum is delivering on and Zappa isn't that makes me recommend Mangum for future uses.

First, Zappa's maintainership hasn't been great. There is constant activity on Zappa, but a lot of low-hanging bug fixes and feature improvements, like the ones I am detailing here, haven't been implemented. Even if it is possible for Zappa to catch up and get these things knocked out the lack of responsible maintainership points to a dying project.

Zappa goes beyond being a shim library to handle WSGI requests and has grown up a small ecosystem of functionality, including the ability to package Lambda zip deployments on your behalf, deploy lambdas for you, create CloudFormation JSON to manage your configuration outside of Zappa, invoke one-off functionality in Lambda outside of the API Gateway request cycle (like a maintenance command), and tail Lambda logs in real time. This is all extremely useful stuff and makes Zappa much more appealing than the bare-bones Mangum. However, Zappa's functionality has slowed its ability to keep up with the latest AWS features. Zappa can only assemble Lambda deployment zip files for x86 Lambdas, not the new, cost-effective Graviton 64-bit ARM CPUs. It also only supports v1 of the API Gateway events, locking you out of a simpler and more cost-effective v2 deployment. Finally, Zappa does not have any way to pass the original Lambda event on to your application code and only extracts the minimum data from the event required to invoke a WSGI request (omitting things such as the IP address of the original request made to API Gateway). This is a really unfortunate one-two punch where Zappa isn't even letting you work around its shortcomings.

If you jump to Mangum you'll have a bit more of a pull ahead of you. There's no tooling to package and deploy a Lambda for you so you'll have to do some DevOps work to create that pipeline. But if you dig in and get it done yourself you could assemble a Docker image for Lambda on the the Graviton CPUs, something not available to you at all with Zappa. Using a Docker image lets you get around the 50 MB limit on zip files for the Lambda deployment. If you are using one of the many WSGI-only web frameworks you may be put off by Mangum's lack of support for WSGI. However, there is an open source WSGI-to-ASGI adapter that will let you connect any existing WSGI code into Mangum, so ASGI-only is not as limiting as it sounds. Mangum is also doing a better job at parsing the API Gateway event payloads, supports the latest version of API Gateway, and even lets you write WebSocket APIs.

Taking the tradeoffs into account, even though Mangum does not offer as much overall functionality as Zappa does right now, it still supports the core, important functionality that you'll increasingly need to use in the future, like 64-bit ARM Lambdas, support for the new AWS APIs, and support for ASGI frameworks. For those reasons I recommend using Mangum for new API Gateway projects with Python.

---

[1] Zappa is named after musician Frank Zappa who needs no introduction. Mangum is named after Jeff Mangum, of the band Neutral Milk Hotel, best known for an album about Anne Frank and "semen stains the mountaintops."