I am an SAP developer since kind of forever, my first job after university (since 2009). First I hated it, but the deeper I am entrenched, the more I see why you choose SAP: Because there really is no alternative. The data model is so complex because it is growing iteratively since the 80s. Hacker News is valuing businesses that adjust to the customer instead of going for some kind of purity. Another complexity especially now is the different paradigm changes in between. ABAP, the programming language of SAP, is quite nice nowadays. CDSViews are a great invention, and so on. There's really a lot of innovation buried in the bad documentation, it's getting better. In summary as bad as it is, I don't see an alternative for all of the backoffice work. SAP is really flexible and supports a lot of different processes. A thing that makes it so costly is a lot of time culture, no small releases, MVP, and so on. But you can be agile with SAP, especially if you move to the cloud.
Speaking of purity, the first thing Hacker News would do is try to run SAP on a Kubernetes cluster, refactor crucial bits in Rust and shard the database into sqlite3.db files on per user basis. We'd also recommend you restructure your office for a more open space, bring down the cubicles and make sure the conference rooms are glasswalled and have funny names. Bringing pets to work is mandatory.
And ironically, the end result would be worse and more complicated, but the people that worked on it would get to write a blog and put it on their CV about it, and move on elsewhere without having to live with their own decisions.
This is so true! As the author of a (small) ERP/MRP SaaS, I am often amused when I read HN and observe the thundering herds of fashion followers — today it's Kubernetes and Rust, Postgres and sqlite have a longer lifespan, but tomorrow it's going to be something else entirely, and you necessarily need to use those :-)
I am always thinking about Vinge's "A Deepness in the Sky" and computer archeology: there is an accumulation of the code base and at some point most code will be some kind of legacy.
It is not feasible to rewrite it in the fancy language/technology du jour - huge waste of resources for things which already work fine and at the moment you are done you have to start again, since the IT-pendulum is traveling in the opposite direction.
We, as an industry are really bad at working with legacy code, but for a lot of us it may be the main task if we endure this line of work for some decades more.
I think there's a bigger issue at play that at some point I want to write on.
Small problems benefit from small tools. Large problems benefit from large tools. However, small problems grow and your bet as a small problem is that your tools will grow with you.
In practice, they usually do. If they don't, you end up extending the tool or rewriting it in a larger tool.
So, our tools start small, and small companies use small tools, and the ecosystems grow together. Then, the successful ecosystems end up the next large problem toolkit.
As you're a small ERP SaaS, you're banking on growing with your customers, I presume. And you can make an argument to start on the large tools at the cost of initial velocity, but that's your tradeoff.
I really think the innovation that needs to happen is a system that has a prescribed growth path. Retool/Excel/similar that can compile to a simple app that can adapt to more complex patterns as the need comes.
Same here, all the hip trends of the day, and then back to the same Java, .NET and C++ that I have been touching for several decades, Boring Technology™.
Yep. If I ever read an article/post about my competition using Kubernetes to run microservices, I'll be quite happy, because it will mean I don't have to worry about them.
> We'd also recommend you restructure your office for a more open space, bring down the cubicles and make sure the conference rooms are glasswalled and have funny names.
This is a genuine question, I don't mean to rip on you: Why is everything SAP I've used terrible? From payroll, employee/student management to industrial control systems - the UX is horrible and it seems so brittle that you're happy if you're out of there as quickly as possible. I've had to reach out to my school's IT multiple times over the years since random parts of the SAP tool they used break.
a) Engineering: Nobody will rewrite 20 year old views just to improve the UX. In many cases, nobody even dares to touch the 20 years old spaghetti code.
b) Sales: The buyer is not the user. The buyer (playing golf with a sales rep) doesn't give a damn about the actual productivity of what he is buying.
c) Management: There is a solid economical rationale in not giving a fcuk about UX. Over the years, SAP outcompeted and absorbed many competitors that were more interested in UX than golf. Golf won every single time. Nobody at the top understands or cares about UX because it does not bring more revenue, it is a cost.
> The buyer is not the user. The buyer (playing golf with a sales rep) doesn't give a damn about the actual productivity of what he is buying
I see this trotted out a lot but are all large companies really that oblivious? Do none of the decision makers talk with the people that work for them? Surely those executives aren't so divorced from reality and common sense that they don't see a problem when people complain that things don't work or if they do, it takes an order of magnitude more time and effort?
Surely there must be some other explanation for the proliferation of these systems other than "execs dumb and bad"?
It’s not “execs dumb and bad,” it’s that execs have different priorities than you think they do, which I think is partially dandare’s point.
Executives at a large company are divorced from your reality at your level of the job.
Incidentally, that’s part of the fun of working at smaller companies, though the variance is higher there’s also the potential to work more closely with good earnest executives who also want the company to succeed.
that's a problem and we're going to have to fix it
>it takes an order of magnitude more time and effort
that's not a problem
the fact that Jane the accounting drone has to do twice the amount of work, or has to hire an intern to help her out, or even wants to quit because the job is so awful now is not a problem
it's a rounding error in the overall cost of implementing an ERP system
a new screen with some new data to show for the exec is worth 20 Jane's and her opinions on the new system
you can't even call the new system unethical or fraudulent because 10 people other then Jane can now stop using broken excel sheets to organize their work
This sounds about right to me. In particular, it tends to partiton the organization into (A) the people whose job it is to plug away at the ERP system (orders, invoices, tickets, etc), and (B) the people who actually do whatever it is that the company does.
(C) - License costs can also play into this partitioning.
Then you hope (and work to ensure) that the benefit achieved for B, by having consistent org-wide information systems to work from, is enough to cover the overheads of A and C.
The simplest explanation is a mis-alignment in incentives.
Executives want to have successful projects to show to their value to an organization. But usually the board members are unable to comprehend the details of a project. So the herd mentality kicks in. The CTO will seek approval for SAP and provide Gartner magic quadrant BS to give the board a level of comfort.
Now whether SAP is the right tool, or if it inconveniences users doesn't matter a whit to the CTO. All that matters is budget and timeline. The CTO wants to be able to speak to the board and say that the project timeline was met and under budget. All else is a complete non-factor.
I hate how bad UX has become the standard due to management/executive negligence, and there is so little the actual users can do about it. Even small players tend to neglect UX because of the financial incentives.
I find smaller companies can have a more direct connection to their users, so the outcome is usually better UX. The disconnect from the user in enterprise software regularly results in poor UX, and the incentives to improve it just don't make their way to the people who need to hear them.
I don't see why you'd split the two. Actual good functionality is UX. Lack of good functionality implies the user has a bad experience given the intended goal.
Big players tend to do the minimum, either attracting with shiny red herrings or barely doing anything at all. Both are bad UX.
From talking to actual end users though I am convinced there is no universally good UX. Someone will not understand it no matter how perfect you think it is.
Now I favor ugly but robust UX for business applications. The user doesn't have to love it, it's their job to use it. It just has to be efficient and comprehensive, not pretty
I am so thankful to be working in an organisation which is reshaping its delivery teams into product teams. Our users are customers and if they want a better UX, they get a better UX.
> It's actually terminal software, that's translated into a GUI from a text interface on the fly
I am intrigued - you mean terminal output is actually parsed, and a GUI view is then generated from it?
E: this would certainly explain the issue of listboxes only having the currently shown items loaded (described by another commenter) - everytime you scroll, the terminal output has to be produced + parsed again
Historically, yes. It was basically like translating an ncurses (semi-GUI command line thing) into a UI. There were attempts to move away from that model, but at least when I worked for SAP, most actual deployments were still text-to-UI. It was basically a way of slapping some varnish on what was fundamentally an old-school mainframe system.
It's the same today - S/4HANA is still accessible from the GUI and DIAG is still used. Web applications (which SAP is pushing customers towards) are freed from this, of course (they talk native http to the SAP application server).
Remember that back in the late 90s SAP had a native GUI that ran on Windows, OS/2 and Unix (Motif-based) and each had their own native controls (a native Windows listbox is implemented differently from a Motif one, for example). Developers would develop a UI in ABAP with platform-independent controls and that UI would be sent over the wire to the client as DIAG and the client would translate that into the native control and data, etc for the end user.
Agreed! SAP's GUI is generally terrible for occasional users (luckily most have now been moved to web-based applications, of varying levels of user friendliness), but in the hands of experienced back office staff it works well. Not pretty, but very functional and with shortcuts for everything.
> the issue of listboxes only having the currently shown items loaded
I challenged this design decision when Fiori was introduced and the thing is that statistically the lists are either small or huge. When they're huge, they got touched in less than 10% cases, so by not loading the whole content you save a lot.
I'm not actually sure, because I feel like most of the things people do with SAP are very repetitive. Their UIs are mind-bendingly bad, but you mostly learn to do the 3 things you have to do all of the time, and then do them over and over again. I'm not sure how much having a nicer UI would make that easier.
It's absolutely not something you just randomly explore. I think that's a more interesting question: what sort of empowerment of employees could you create if the UI wasn't so terrible?
But SAP's moat is elsewhere. You put up with its terribleness because nothing else can do what it does. Beating SAP would require you to reach that, and do something else, part of which could be a nicer UI.
Every business software is at least as complex as the business process it models. That's the thing you training for.
Every detail of the business process is now explicit. Employees have to be forced to work according to actual process. You need quite a bunch training for that. Then employees will find shortcuts to make their own job a bit faster/easier, which will fuck up some cases. You need even more training for that and possibly some redesigns. And all you get from that is depression
No, because the training is about the core functionalities of those ERP systems, what action triggers what and so on. Training is not about how the UI works, this knowledge is gained on the job.
They wrote you can be agile if you move to the cloud. Did you try that?
I jest. But that's kind of the point. SAP is terrible by the nature of the beast. It's a closed off system with specialised developers who require all sorts of expensive certifications. That doesn't make for good developers, that makes for pigeon-holed developers who don't have a lot of competition.
A terrible SAP developer with all the certifications to their name would probably still find plenty of work, because the expectations are low to begin with, as proven by SAP being held in low regard across the industry.
To me, needing expensive certifications to prove your worth (as if...) is a big red flag. I'm a developer who has 20+ years of experience, I recently worked for Apple and other Fortune top 50 companies, I went from startups to enormous companies.
Nowhere did I need certifications. And my past experience was never enough to land a job. I'd have to prove myself in every job application. That's tiresome and feels extremely unnecessary, but it requires me (and my peers) to stay sharp.
Of course, none of the above is very black and white. There are certified developers who are amazing, and there are open-source developers who keep themselves relevant who actually suck at what they do.
But I'd argue that the SAP group of developers have far more developers who aren't very good and grow complacent, oftentimes because of their certifications. That, combined with a closed-off system, bad documentation, a lack of online support, and a much smaller community, will MORE often lead to software that is of lower quality.
I think you don't see the full picture here. The developers working for SAP usually don't need any certifications. They are regular computer scientists/physicists/mathematicians etc. who write code / design systems.
There are of course the certified consultants that work for clients of SAP. I can however understand why they need certifications. It's probably impossible to configure these systems without some kind of special education in them.
And as for the quality of SAP devs.. I work for SAP and I have met many very qualified developers at the company. Far more than I anticipated before joining the company. Quite a few have been poached by Google et al but that is another topic.
I think they were talking about "front end" SAP devs that are hired to work on custom solutions by a client. Not the core devs that actually work for SAP on building the product itself.
I think ERPs are a basket case category generally. I've worked with SAP for a few years (or more accurately work hard to keep it out of scope for what I do) in my day job. I've also worked for a company providing ERP stuff for SMEs (mostly S).
Anyway SAP/ERPs aren't code for running your business. They're code for running code to run your business. Now you have all the shortcuts SAP made to get stuff out the door, and on top of that you've got all the layers of shortcuts your business has made to get things out the door too. Therefore lots of nasty difficult complexity.
And finally I've seen evidence that in SAP people treat it like the fundamental abstraction layer is the spreadsheet[1]. So like in unix everything is a file, in SAP everything is a spreadsheet. This is a nasty complicated fundamental abstraction without the natural elegance of Unix's one.
[1] Maybe it really is, maybe it isn't but that's how a large chunk of the ABAP code I've seen treats it.
I remember SAP circa 2003. The only think that used XMLHttpRequest (AJAX aka dynamic html) was Outlook web frontend, and SAP web UI. GMail that widely popularized dynamic html did not even existed yet.
"SAP developers" can mean two different groups of people. The developers who build SAP, and the developers who build the custom stuff that clients need. The former can be quite innovative, but the situation with the latter is pretty damn grim (and I say this having worked in the industry, and my dad still works on it)
The UI isn’t just terrible, it’s hostile to the user. The rare times I am forced to interact with it I feel like I’m using software that was meant for some other purpose. I’ve worked in the industry for 25 years, I’ve never come across worse UI.
My favourite bit of the SAP UI is whenever you scroll a listbox, it goes back to the server to load the rest of the list. It only has data for the exact 5 items it's showing; the other 1 above or 3 below each need a server hit. And it forgets the data it loaded before.
It isn't use for you as a user, it is meant to run your company. You are not the user, you are the one feeding data into the system. And that is actually perfectly fine in case of ERP systems.
It definitively is neither pretty, discoverable nor intuitive, but in the hands of experienced power users (who have spent a lot of time learning the UI), it works really well. I have seen people (who have used the system probably for > 10 years) dig the most arcane information out in seconds, navigating through about ten different views without ever touching a mouse.
It's a UI from another era, but if you take the time to learn it properly, it can be very efficient.
I kinda feel like SAP is the ultimate in checkbox checking. If you ask every department what they need, and write it all down, then find the product that matches all the checkboxes, SAP is it.
SAP checks most of the boxes of almost all industries you need to run a modern business. Some specifc domains better than others, sure, but none are actually bad or unusable. And all tjose functions are integrated with each other. That alone is tremendous value.
Some weaknesses I see with SAP compared to dedicated solutions are in logistics and warehouse management, process industries (e.g. continious chemicals production, SAP is way better for discrete products and still it works just fine for the pricess industry), aerospace MRO (SAPs aerospace and defence package is more geared towards production) and eCommerce (eCommerce as a business is not that complex).
Basically, if you are a manufacturing company SAP is easily among the best solutions out there. Added benefit of using an ERP that litterally everone else is usong as well: you have a way larger pool of people to recruit from that know the system. The less market pebetration an ERP has, the harder it is to get people who prwviously worked with it. That means higher training efforts, more need for external cobsultants and performance issues until your emoloyees adapted to whatever ERP you have.
I think part of it is the problem space to be honest. SAP basically gets used to glue all the fiddly bits of organisations together and I don't know that there is an approach to that set of problems that doesn't get tangled up in corner cases.
With a lot of the nice looking web based apps we're used to looking at, the UX and the target domain evolve together and if there isn't an elegant way of doing a particular operation then there's going to be pressure to drop that operation entirely. SAP deployments always have to do everything in the organisation so complicated corner cases either get dropped (bad) or end up with suboptimal interfaces (also not great!).
Why did AWS, a group probably fairly technical savy and a direct competitor to its vendor, take so long [1] to migrate from Oracle?
Why does Google, as of today, have job openings for SAP in its Rev Rec division [2], and more widely in areas touching [3] “ SAP ERP domains (e.g. Finance, Revenue/Cost Management, Billing, Materials Management, Sourcing, Procurement and/or Inventory Management)”?
Granted you could say, and I believe in one of the many google “corporate biography” books Eric Schmidt allegedly worked through this same problem: hey is an ERP a product for us? A core competency? Should we build it? [Seems HN discussed this in 2021 already, 4]
Conversely, given the wide ranging scope of random things Google has built and how naively simple accounting and something like a fixed asset ledger looks at first glance…why did they never do it? Surely not because building 7 conflicting chat tools was a priority.
My guess (as a non developer) would be it’s crazy hard to build SAP. From personal experience even QB Online has a daunting level of “backwards compatible” complexity [5] in its data scheme even coming from an accounting background. The API keeps versioning up incrementally and you’ll find gems like “XYZ local French Tax” and other accumulated baggage. As an anecdote, using arguably a knowledgeable vendor, as of a year ago it wasn’t possible to populate a full simple profit and loss statement via Fivetrans ETL tool [6], even though they did a phenomenal job in mapping out the ERD compared to anything else that existed imho [7]. CDATA let you run SQL queries, but the complexity of scripting some of the reports was much more fragile. The community even built a DBT layer [8] and given how cumbersome it was to generate even just the “Revenue” line on the P&L out of a simple non-enterprise tool like QBO, SAP seems 1000x harder.
That’s probably why everything in-market, post-sales cycle is garbage. But hey, someone in a dorm room might be working on replacing SAP right now :)
Note: this excludes versioning, which I believe Workday does every six months, which is also a bunch of work twice a year.
> ABAP, the programming language of SAP, is quite nice nowadays
I took a brief look at the ABAP wikipedia article and this insanity [1] stood out to me.
I think after a decade of SAP you are too brain-washed to give an objective opinion anymore.
Wow. That is pretty insane. I hope there is a tooling to surface a warning when it detects this.
But lots of domain specific languages have idiosyncrasies. The Apex language used to program Salesforce is a half-implemented clone of Java from a decade ago. And they didn't cherry pick the best parts.
Some companies will be forever scarred by the past trend to create an entirely new programming language for their product when an existing one could‘ve been entirely sufficient
For a regular software developer, ABAP is pure insanity. But it has powerful data manipulation procedures, it's easy to just create and manipulate all your data tables in any way you want.
>, but the deeper I am entrenched, the more I see why you choose SAP: Because there really is no alternative. The data model is so complex because it is growing iteratively since the 80s.
Yes, the scope of ERP and SAP is so immense that even other software companies that have competency in the business of programming also buy SAP licenses instead of coding their own home-grown ERP system.
E.g. Microsoft, Apple, Amazon, Google all run SAP.
Microsoft even has their own ERP software (Dynamics GP acquired from Great Plains Software) and yet they run SAP. Microsoft sells Dynamics GP to mid-tier businesses but they don't use it internally for the consolidated financial accounting of their complex
multinational business.
And in an interview about Google's early days, one of the employees recommended that they install ERP system like SAP and co-founder Sergei Brin was skeptical saying "Why would they need to buy that when their programmers could just code it themselves?!?" Well, Google did eventually buy Oracle Financials ERP and then eventually switched to SAP: https://www.google.com/search?q=google+migrates+oracle+finan...
If you're a brand new YC startup, you're not going to need a complex behemoth like SAP ERP. Maybe Quickbooks or some SaaS service for accounting & HR/Payroll is all you need. However, if the business grows enough (i.e. multinational), you'd be wasting money by paying your own staff programmers to re-invent what SAP already does. Just pay the millions in SAP licenses. It's expensive but still cheaper than trying to do it yourself.
After participating in the hiring of an SAP developer I realized -- financially speaking, anyway -- that I picked the wrong thing to specialize in.
The first time we brought in someone truly experienced in SAP development, we paid him more than any other developer on staff[0]. I appreciate the different take on SAP. Can I ask -- is it still as lucrative to be an SAP developer?
[0] This was a global multi-national telecom who ran an in-house developed audio conferencing service (written entirely in C++, interacting with a lot of hardware) ... the skillset of some of our devs was incredible, which made this all the more surprising.
SAP developers aren't paid because they're more talented than other developers, they're paid because the niche in which they're deeply embedded extracts extreme business value.
It may take just as much talent to write interfaces for obscure audio hardware and deal with janky bluetooth edge cases, but that doesn't mint the same $$$$ as a SAP guru who refactors a business process to be more efficient.
Depends what you're comparing it to. Even in Germany it's a niche skill set, that has a higher payment than the average Java guy, but with enough skills you can get paid the same for any specialization, e.g. security, cloud-computing. I changed to solution architect recently, which is a bit removed from SAP work itself.
I am a project manager and a SAP user. First I hated it. And the more I used it, the more I hated it. It's like a really, really uncomfortable chair, strong, but designed with zero attention to ergonomics.
Hey, I worked for SAP for ten years and I don't agree with you. ABAP is still ugly with all its improvements, version control system and developer experience is awful. CDS views are great until they aren't, then we had this newest feature where you build apps with metadata on top of CDS views and that's where I decided that it's not a software development anymore.
I do see how Fiori launchpad is a great thing, how HANA is extremely powerful if you know what you're doing. At the same time I couldn't stand the corporate culture, QA processes and etc anymore. There're innovations and innovative teams but for the most things SAP is doing, software development is not the core competency.
Fun fact: I work with Datapath now (if you know what it is then you get an irony)
This should be the SAP tagline. I swear I've heard this since I started working with SAP in maybe 2009(?) as well - I had a brief stint as an ABAP developer.
Indeed, for a long time it seemed like SAP didn't "get" cloud. Having to sign long contracts and speak to an account executive to get access to their cloud platform (rebranded every few months, now "Business Technology Platform", BTP) was the way they did things. Contrast with Google, Amazon, etc.
I've not yet used S/4HANA Cloud (only the on-premise version), but it looks to me like it's just a hosted version of standard S/4HANA but with various restrictions on what you can customise etc. No real elasticity, cloud scaling, etc. SAP ERP has been multi-tenant since at least R/3 in 1992 (I never used R/2) so "Cloud ERP" smells to me like "we host it for you and call it cloud".
There are two versions, the on-premise landscape and the cloud version. The on-premise version makes sense sometimes. You can even deploy SAP platform to cloud foundry.
maybe this one: https://open.sap.com/courses/abap1 In general open.sap.com is the site where you can learn stuff. but depending on your customer's stack it might be different.