How do you ensure the data infrastructure you’re building doesn’t get replaced as soon as you leave in the future?
If this is a core conceit of the thinking then my answer is who cares?
Why do you want to try and influence a situation you're not even involed in?
Taking it back to the best lesson I was ever given in software engineering "don't code for every future".
Do what you're asked to and not get caught up in projecting your own biases into trying to make a "solid base" for the future when you can't know the concerns of said future.
No matter what you build someone will come along later and try to rewrite. If it is built too well with too many future cases in mind it will be too complex. If you write something simple and basic someone will try to add their complexity. If you write in one language someone will try to use something different. Same goes for framework.
Write for your current requirements not some future state because people will say your work was subpar or overkill regardless because you are not around to defend your decisions and putting you down raises them up.
The desire to rewrite something is the single biggest red flag for me that someone has questionable technical decision making skills. Yes there can be good reasons for it, but my priors shift dramatically once I hear someone suggest it
I used to agree, but there’s so much software that is just so bad. Wrong DB, bad framework, crazy abstractions, black box magic, no security, behavior that is just wrong, etc.
The dev team going away for 3 months to rewrite is probably a bad idea, though. There are definitely good ways to rewrite and then not good ways.
I'll agree with you if you mean - entirely en masse. I specialize in fixing legacy software systems that have ground to a halt development-wise. Blanket rewrites are almost never a good thing, but partial rewrites are a wonderful tool. Like most things in this industry there unfortunately isn't a hard and fast rule. Everything is extremely context-dependent.
Some data retention requirements are mandated by law and it is necessary to develop robust systems that can stand the test of time. I've seen 15 and 25 year retention periods for data in safety related applications.
Things my interns learned in the first month as part of new hire training.
My quip above is to illustrate that in a dynamic and complex field its important we don't over index on experience.
Then let someone come along and try to rewrite and improve - but if your solution is so flimsy it forces a rewrite, it's just poorly made to start with.
> If this is a core conceit of the thinking then my answer is who cares?
Yep. At the end of the day, it's very simple:
People working for a company are not ants or bees. A company is not a hive and people are not going to put down their own interests to serve the hive. We are a bunch of cooperating, but ultimately independent agents, who act in their own benefit.
It is up to the business owner to keep their employee activity in check. Does that mean giving them work to do? Checking on the progress of their tasks? Checking on their methodology and software stack sustainability? Making sure there are no single points of failure for the business? Making sure the "IT know-how" of the business is preserved when a person leaves? ALL OF THE ABOVE!
When a business owner can't do these periodic checks themselves, they're free to hire someone that will do this for them.
But the idea that individual developers should care about what happens to the business after they leave is just preposterous.
Also, the entire "resume driven development" thing is absurd. This has always happened in software development. People care a lot about what their resume will look like in 5 years. It's perfectly normal and the business benefits too ("we use modern tools, come work for us"). It doesn't mean the business should allow needless "shiny new thing" syndrome to thrive, but you should watch out to not stomp out innovation or you might find yourself unable to hire talented devs because no one wants to work on your shitty "php with jquery" web app.
> But the idea that individual developers should care about what happens to the business after they leave is just preposterous.
It's not about caring after you leave. It's while you stay caring enough to do useful things for the company. Sure, you can be like a consultant (require very specific requirements and not trying to understand or put things in perspective), but as an employer these are the first people that I will let go because they bring less value than someone that "cares" (again, while being there, not after they left)
Yes. Put another way, this school of thought concerns professionalism while you're there, when you already know that what you do will still have effects after you're gone.
A different school of thought is that a job is about showing up and doing some interpretation of what your your manager tells you to do. This might not be very aligned, and much of the org chart might not be very aligned, so the priority tends to be appearances. Manager told you to make a Web site that does X, so you try to make a Web site that arguably does X. You don't tell the manager all the factors that in a better organization they should care about, and you maybe don't do a particularly good job of the site you do make, and you definitely don't base all your implementation decisions based on company needs rather than your own resume and political capital. But you're satisfied that you arguably did what you were told to do, and that's the transaction.
The latter school of thought is very common, and I think it's not really due to individual ICs. Rather, usually the organization is actually pushing people towards that thinking, because the org chart and practices are also full of that kind of thinking. A more conscientious professional would blow a gasket, due to the "preposterous" situation of a company of individual irresponsible mercenary behavior and collective dysfunction like that.
I naturally subscribe to the true alignment school of thought, and that's one of the appeals of being a startup founder: I can apply my experience (and, admittedly, just as much theories/guesses) towards building a company and team where things are aligned better. It's also one of the reasons I dread some aspects of founding, because I know that, no matter how good I am about hiring and onboarding into the aligned culture, we'll sometimes have to deal with very mis-aligned (even bad-faith) people from partners/customers/investors. Not only is that unpleasant, but there's the risk of infection.
Being an individual developer that does think about those things, even if they are not actually doing them but at least helping with all those checks, is a strong differentiator for promotion and higher pay.
As an employer you _want_ these kinds of people around, helping you with process and making sure if they leave things are still functioning, thus you have more insensitive to keep them around / pay them more.
So it is in the best interest of the individual developer to work with those goals in mind too. Yes it is not your responsibility, but it can be, and that can give you more leverage in salary negotiations.
So its always good to think about “cui bono” and be sure you’re on the right side of each advice :)
SaaS was born from comments like this. Paying to keep the lights on, effectively ensuring that an employee quitting won't undermine the entire operation.
SaaS companies simply cannot be trusted to not leak customer data. They always will leak it to hackers. This is different from major clouds and self-hosted services which have different sets of security considerations. Snowflake validated this assertion this year with a major data leak.
Also, with SaaS, you pay 5-20x for everything. For example, you can self-host Airflow in a $20 USD/month VM, but any managed Airflow service is going to cost astronomically more.
A lot of the article relates to a key person dependency issue.
>Sure, they have built data infrastructure that works and solved the businesses current problems. They maintain it, and no one asks questions.
Probably all the "budget" has allowance for is current needs. Some of the key engineers may not even be paid very fairly considering the true magnitude of business problems being overcome currently. You can't really expect them to prepare for a longer future than they have already been fully staffed for, especially succession.
>Perhaps no one realizes that one of the team members has to wake up at 6 AM every morning to check and ensure all the reports and tables have been created.
>That all works until the day they leave.
If a talented engineer is regularly working overtime to get things going like infrastructure, or worse to keep things going, even worse to keep things from failing, then that engineer is definitely short two staff members. And has probably been short the entire time. Nothing less than an assistant engineer and a technical secretary if they want real documentation as they go along. Plus even more true investment if there's any need to make up for lost time.
If infrastructure is important, something like this is absolutely pure executive failure from someone who's just not in the proper league.
You can not paint a pretty picture, and it's reported to be very difficult to fix stupid.
Some people just should not be accepted as executives in technical endeavors.
It can be tough for lesser executives to accept a non-cutthroat non-business-ladder-climber as more of a "key person" than themselves, but it is far too often the case. Whether the bonehead executives realize it and shrewdly calculate how much more payroll would expand if there was to be better coverage, or are completely oblivious, as the article says about the overworked engineers:
>They maintain it, and no one asks questions.
Example article is from a more expert data "repairman" who knows better than to rely on a single company as an employer if it's got dingbat executives.
There's so many of the under-qualified executives to go around, he's got a lifetime of work ahead of him as a consultant fixing the lackadaisical way they let technical debt underlie a business to where it could topple unexpectedly.
You should care because, the vast majority of the time the person working with it after you, will be you.
But to your point, people think they want “flexibility” or some similar concept, and they end up adding immediate complexity that never pays off, or worse, they pick the wrong abstraction and have a mess that’s hard to undo later.
What they should be aiming for is simplicity. Instead of trying to predict future, keep it as simple as possible to give that future person a chance of tackling the future needs when they arise.
You can view the question as a proxy for "how do you provide value for money?".
If you build something that then gets replaced a few years later, maybe you did something wrong. Ideally you make something that evolves, or even better, that acts as a foundation others can build on. If you get a lot of assumptions right and the implementation doesn't get in the way of what people do - or better yet, meaningfully enables them to get work done, you've succeeded.
Here are some things I've observed in the wild.
Data infrastructure projects often fail, not because the technology doesn't work, but because the solution does not enable _organizations_ to work with them. I've seen many companies invest millions in solutions that eventually turned out to be useless because they failed to help make data and results accessible to complex organizations with lots of internal boundaries.
Too much too soon and too complex. You try to address every possible need from the start and in order to make the feature list as long and impressive as possible, you introduce lots and lots of systems that are expensive and complex. Then to use the system, you unload a huge burden onto the users. They have to learn all of these systems and spend lots of time and money training people and adapting their systems so they can interoperate with the rest.
I've helped a few companies design their data infrastructure. I usually follow an extremely minimalist approach. Here's how I start.
1) your long term data store is flat files, 2) you make real-time data available over streaming protocols, 3) by default everyone (inside the company) has access - access limitations have to be justified, 4) you document formats and share code that is used to interpret, transform and process data so the consumer can interpret the data. 5) you give people access to resources where they can spin up databases and run stuff. Data producers and consumers decide how they want to create and process data. You focus on the interface where they exchange data.
(I left security as an exercise to the reader because a) it depends and b) how to secure these kinds of systems is an even longer post)
Points 1 and 2 are sufficient to bootstrap databases and analytic systems at any time. Including systems that receive live data. It makes it possible to both support systems that are supposed to be up permanently and systems that perhaps only load the data, do some progressing and then get nuked. 5 provides the resources to do so.
3 usually meets with resistance in some types of organizations, but is critical. I've seen companies invest millions in "data lakes" and whatnot ... and then piss away the value because only 2-3 people have access to the data and they ain't sharing. You need executive management to empower someone to put their foot down. (One way to make people share data is to use budgets. If you don't share data, your department pays for its storage. If it is shared, it is paid for by a central budget.)
Point 4 requires you to also educate people a bit on data exchange. For instance in many areas there exists exchange standards, but these are not necessarily very good. If you find yourself in a situation where you spend a lot of effort expressing the data in format X and then spend a lot of effort interpreting the data at the other end, you are wasting your time. Come up with something simpler. Not all standards are worth using. And not everything is worth standardizing - don't lose sight of actual goals.
Point 5 is where you grow new core services. Producers and consumers get to pick their own technologies and do whatever they want. When they can show that they've built something that makes life easier for other parts of the organization, you can consider moving it to the "core" but this only happens when something has shown that it works and improves productivity across internal boundaries.
If this is a core conceit of the thinking then my answer is who cares?
Why do you want to try and influence a situation you're not even involed in?
Taking it back to the best lesson I was ever given in software engineering "don't code for every future".
Do what you're asked to and not get caught up in projecting your own biases into trying to make a "solid base" for the future when you can't know the concerns of said future.