This solves a really tricky Terraform problem I have at the moment about wanting to create Dev environments quickly but not wanting to recreate DNS, VNets and other core functions for every user. Will definitely be having a play with it.
Really well put on the risks and how outcome based projects aren't necessarily better aligned for good outcomes.
The best company I worked with did a multi-drop "agile" contract where there were several incremental prototypes and product releases with feedback and tuning the requirements to ensure we got what we needed in the end. That felt far more aligned and how I'd prefer to work but it took a lot of trust and a long relationship with the vendor.
I feel that it's almost always necessary to have more than one delivery at different stages otherwise some important feature or bug fix doesn't make it in.
You could say each increment is a fixed partial fee or milestone but that doesn't work unless there is adequate requirements analysis at the start, and they are small increments that can be determined dynamically and/or in a highly prioritized way.
And also each increment has to be the equivalent of billing for that block of time anyway.
I love the idea of doing independent technical due diligence work after enjoying doing software auditing and investigations in my current job. Looking at Gitlab's processes it seems like company structure, finances and legal checks make up a lot of the investigation. The software/infra/tech side seems quite short (Gitlab suggest their early technical diligence should be turned around in 2-3 days).
Do you think there's enough demand out there to have a company that does just technical (e.g. software quality, security) due diligence?
> Do you think there's enough demand out there to have a company that does just technical (e.g. software quality, security) due diligence?
My company (used to, we've grown to do other things) only do Tech DD and we've done really well, so the answer is yes. According to Pitchbook there are about ~3,000 software company acquisitions every year - so that might tell you how much potential business there is.
There are consultancies who provide these DD services certainly as a line of business, if not their entire business. Like any other professional services enterprise, a lot of it depends on your network and ability to generate new business, especially in the early days. If you're not in the position to do it yourself, perhaps there are opportunities to get involved on a contractor basis with firms that do.
I always enjoy seeing what a good awk & sed user can achieve in bash.
However one of the not so good Devs I worked with used awk to load a large, deeply hierarchical JSON file. They refused to use a library to parse JSON. It was a many hundreds of line monstrosity.
Luckily when they left we were able to parse it in JQ instead .
I think Gitpod is a great product, and I've had a massive amount of use of their free tier. There's something pleasant about throwaway Ubuntu development environments. I can cover pretty much all of the major development use cases in it, even container work.
I just hope they survive long enough to take off as they're giving a lot away for free currently. Their dedicated instance setup looks like it'll cover the majority of businesses who'll likely have their own landing zones and cloud controls they can integrate into. Not being able to self host is a loss, but it might the right trade off.
I don't understand the AGPL bit well other than it was seen like poison by corporate risk types at my previous jobs.
I remember a colleague who was critical and pretty much the only person who knew how part of the business worked. He had a great analogy that for every new initiative and fresh recruit they brought into that area, it was like the organisation was hooking in another IV into him to draw more blood out. It's an image that always stuck with me to describe people in the situation Jeff found himself in.
As someone whose job involves maintaining uptime of a critical system that's dependent on Cosmos DB this sort of thing is scary. Where there's been other reliability issues with Cosmos before we've not had an understanding customer base, and it feels very out of my control.
I'm finding a lot of the reliability guarantees of Azure PaaS services are overblown or come with big caveats when you start to work with them in a serious way. For example I've had some bad reliability issues with Azure Functions not firing, or their premium function runtimes becoming unresponsive. And it seems like that's just the start of the outstanding issues with them https://github.com/Azure/azure-functions-host/issues
I think people need to look more carefully at these PaaS guarantees and look at what that 99.999% reliability Microsoft are claiming actually means.
After using AWS for 3+ years and GCP for about 6 months, I can say Azure significantly lags behind them. Their service reliability is astonishingly poor. I think our most recent issue was 67 VM failures in a VMSS (of 55 nodes) backing AKS (Azure Kubernetes Service) in a single month. The health events said there were some kind of "remote storage errors" making the VMs unhealthy
That's a couple months after the Ubuntu/systemd incident (Azure's "blessed" Linux image is Ubuntu and it has unatttended-upgrades enabled including on managed infrastructure like AKS (where you can't turn it off without dirty hacks). A bad Ubuntu update caused hosts to lose their DNS from DHCP config rendering massive amounts of machines in partially broken states)
Do you know what blew me off? When azure executes maintenance on for instance postgresql servers, there is no record of that activity in the activity logs or anything to note in service health. The service was unavailable during the maintenance. And stronger yet when the database is unusable due to an incident the cpu is maxed out and it doesnt allow any successful connection, nothing is detected.
How can this be a premium iaas/paas? Azure feels like the MS teams of tele conference. Companies buy in because they are already in the MS world. Not because azure is better.
Yeah, that's the one we've had a lot of problems with.
> And stronger yet when the database is unusable due to an incident the cpu is maxed out and it doesnt allow any successful connection, nothing is detected
Apparently Azure's storage system that backs this uses some sort of thread pool and the thread pool can lock up/become exhausted leading to I/O starvation. When this happens, connection attempts fail. When the connection attempts fail, it can lead to a connection storm where all these new connections rolling in exhaust the CPU. The telltale indicator is Postgres checkpoints getting behind.
All the while, the DB I/O metrics look like they're completely fine because it's not hitting an I/O limit, it's hitting thread pool exhaustion in the some storage system under the instance, outside of Postgres.
You can also get some clues if this is the problem by enabling Performance Insights and checking the Waits tab. If all the top waits are related to I/O activity, that's another dead giveaway the storage system is locked up again. You can just web search the name of the waits to see what causes them. AWS has some nice docs detailing Postgres waits
Thanks for the detailed explanation! We didnt look into this so detailed yet but what you are describing sounds familiar.
Since we have premium support (P1?), we had some internal azure postgresql engineer look at the issue and they pushed the problem back to us. Blaming our app not built correctly. That has been ping-ponging for over a year now.
Finally i saw this semi-acknowledgment in their health status yesterday.
Do you happen to know a proper solution? Are you waiting for them to fix this issue or moved to a different db service?
We've talked to the Postgres product engineers many times. Proper solution is to run away from Single Server as quick as possible. Flexible or Citus Hyperscale may be good solutions. We're currently using Patroni to manage VM-based clusters (but still have a lot of data on SS)
Personally, I'd look into a 3rd party if you want managed Postgres (assuming you don't have contractual obligations that might complicate 3rd party access). There's vendors like EnterpriseDB, Scalegrid, etc that provide various solutions (I don't have any recomendations here--Postgres has a list of managed providers by country https://www.postgresql.org/support/professional_hosting/nort...)
The hard part for us is figuring out how to migrate away from single server when it's used in production. It takes eternity to migrate data away from the thing, we are looking at ~24 hours just to get data out, and then we need to figure out how to do a live cutover or backfill.
Absolutely agree on a third party. Azure is just a let down overall.
The gods are angry i think. Woke up today and all our pg servers were unavailable. Checked service health and azure shows a global pg incident impacting pg servers.
And the funny thing? Status.azure.com is all green. No events in activity overview. No service health within the affected instance.
Workaround advised by azure? Upgrade to next plan. We already reached the maximum size. Maybe time for Citus . More $$$ for M$
> I think people need to look more carefully at these PaaS guarantees and look at what that 99.999% reliability Microsoft are claiming actually means.
Hypercloud managed service SLAs: all the fun of novel complex, technical solutions in production + the transparency of cast iron + the pendanticism of being a contract lawyer
Which leaves exactly zero people who are excited to be at that intersection.
This is a cool project, and really well presented.
I've looked doing similar, trying to query a large dataset through ranged lookups on a static file server in the past. I ran into issue of some hosters not providing support for ranged lookups. Or the CDNs I was using having behaviour like they only fetch the origin data source in 8MB blocks meaning there was quite a lot of latency when doing lots of small reads across a massive file.
It would have been interesting to find out a bit more about these topics, and see a bit more on performance.
I, too, have used Typora for years whilst in Beta. I appreciated it's approach and features enough that I've paid for it.
Alternately, there's a free editor called MarkText that is maybe 85% there as compared to Typora. I use this on my work Windows machine, which is pretty locked down due to corporate policies.
Typora is fully offline and works on real files, there's no cloud dependency here AFAIK. At most you can keep files in your Google Drive/OneDrive/Dropbox syncing folder to make it "cloud".
This tool has reminded me a few years ago I created a helper web utility that let me search Python libraries and get a tree view of their dependencies, and some license info. I had to do a lot of manual Python library compliance before we had tools like Blackduck.
It's got a few shortcomings. Dependency resolution in Python is pretty difficult to work out when you've got a lot of libraries with common dependencies. And the license info on Pypi isn't always correct. But it's always been a quick useful tool for me.
