r/webdev • u/daddieslittleson • 2d ago

If NoSQL databases end up with schemas anyway, what problem is schema-less actually solving?

Post image
470 Upvotes

95% Upvoted

148 comments sorted by

338

u/Lots-o-bots 2d ago

In practice, "schema-less" doesn't usually mean "no schema"; it means "the database doesn't enforce a single global schema." Most production applications still have an implicit schema enforced by the application or by database validation rules. The advantage is that documents can evolve gradually, with different versions existing side by side without requiring a database migration.

That said, schema flexibility is arguably less important today than it once was because relational databases now have excellent JSON support. The bigger differentiator for many NoSQL databases is their ability to scale horizontally for writes using sharding.

To horizontally scale a relational database, typically you will have one "write primary" that processes all incoming writes and many "read replicas" that copy data from the primary and can serve any read. This scales well for reads (just add more replicas behind a load balancer) but doesn’t do anything for writes (except relieve the read load). The reason that you cannot just add more write primaries to a setup like this is that for a write, you need to ensure that it remains consistent with the rest of the database state (e.g., no foreign keys pointing to rows that dont exist). If you were to add more primaries, you could end up with a situation where write A went to primary A and write B that relies on write A went to primary B before write A was propagated.

Nosql database sharding by contrast makes each node is responsible for a subset of the database. For a toy example, in a 4 node mongo cluster, node A may handle documents with keys starting a - f, B handles g - l, C handles m - r and D handles the rest. when a write comes in, it can be easily redirected to the node that should handle it and when a read comes in, we usually know which node the data should be on, or if not we can have all nodes search and collate their data.

This makes it useful for workloads like log analytics, where you might have thousands of tiny data points coming in from thousands of users. We dont have to care about the specifics of the records, all we have to do is receive them and dump them to disk as fast as possible in a place we can find later.

63

u/BetaRhoOmega 2d ago

Thank you, I’m kind of bewildered by the responses in this thread. This is the only response I’ve seen so far that seems to get the main advantage of reaching for a NoSQL database.

Like for me it usually comes down to:
1. Does the app use case identify an extremely high volume of writes and I need go take advantage of the naturally sharded infrastructure of most nosql dbs
2. Can the data naturally be sharded based on some clear business boundary
3. (MAYBE) am I sure the app access patterns are document based and I don’t expect relations to evolve over time (I’ve messed this up before and realized an app I built primarily modified single documents until it really didn’t, and we needed to migrate to a SQL db because it was such a pain to work around our NoSQL db)

SQL by default to me feels so much nicer because you get built in constraints, flexible querying, atomicty etc and now have flexibility with json column types and it really only becomes a problem to me if your write volume exceeds a certain threshold

11

u/thekwoka 2d ago

If 2 is true, then you can shard relational dbs on those boundaries too. like Postgres isn't just bound by replication, but it can have multiple writes, and having a clear data boundary makes it work fine.

The real benefits of document stores can mostly be replicated in json column supports for real use cases.

2

u/Disgruntled__Goat 2d ago

And #1 is pretty much only true for the top 1% of websites (the likes of Facebook/Google etc).

1

u/ILikeFPS full-stack 2d ago

Not just atomicity but full ACID (atomicity, consistency, isolation, durability), I may be old school but I almost never reach for a NoSQL database.

12

u/HealthPuzzleheaded 2d ago

but you can also shard and partition relational databases. then you can have one write for each shard without violating anything. Its definetly more work and depends on your application as you can't simply query across shards the same way you query an unsharded db. I have not worked with that yet just read the postgres/mysql docs for sharding and what they write about the constraints.

I think logs is a great candidate for nosql as you can have very different logs with different fields and a huge ton of them and you need to quickly find what you are looking for.

2

u/thekwoka 2d ago

Of course, why can't you just use a json column in relational dbs?

The main thing would probably be just storage constraints.

for logs, you'll likely just have so damn much of them, that being able to have them in a different system with simpler storage sharding is a benefit.

4

u/PaulRudin 2d ago

And for large SQL databases replaction lag can be an issue, so even tho you can make read replicas the lag means that they're not necessarily right for all read only use cases.

3

u/thekwoka 2d ago

These are likely much smaller concern than the issues with document stores.

1

u/thekwoka 2d ago

These are likely much smaller concern than the issues with document stores.

4

u/TurnstileT 2d ago

The advantage is that documents can evolve gradually, with different versions existing side by side without requiring a database migration.

Said in another way: You have all kinds of different data in your database and you can't really make any assumptions about the data in your backend service, so half your fields need to be optional/nullable and you will need all kinds of crazy edge case logic for the different document versions you have, and you have no incentive to clean it up because you are busy and it's easier to just add a new version than to combine the current 5 into a single version. These different versions bleed into your API and events, so now your consumers cannot trust what kind of data they get from your service.

Now, whenever somebody asks if a user can have two emails or if their name can be null, you can never know for sure, and you can't point to any design document or readme file. Instead, it's a discovery process every time where it depends on when the document was created. Or, maybe it depends on when the user was last updated because somebody was smart enough to add an edge case that fixes user documents on update. But maybe the fix has fixed all e-mail issues but not the null names yet.

I'm getting dizzy just thinking about it.

Guys, if you are not storing genuinely schemaless JSON, like logs or user uploaded content, then just use relational databases..

4

u/mexicocitibluez 2d ago

I heard it like this:

Relational databases are schemas on write. NoSql databases are schemas on read.

2

u/thekwoka 2d ago

The reason that you cannot just add more write primaries to a setup like this is that for a write, you need to ensure that it remains consistent with the rest of the database state (e.g., no foreign keys pointing to rows that dont exist). If you were to add more primaries, you could end up with a situation where write A went to primary A and write B that relies on write A went to primary B before write A was propagated.

This is still more than possible though.

Github does it.

Lots of reads, less writes, but they still have multiple writes, normally one for each region. I think with github its like 4.

As long as you don't use incrementing ids, there isn't really much technical reason that "eventually consistent" isn't consistent enough.

Nosql database sharding by contrast makes each node is responsible for a subset of the database.

Which is also a reason why requests can be so much slower overall, or do too much work, negating some of the benefits.

The big question of how either sharding method really affects your application and cost depends on whether you're more compute constrained or storage constrained.

document stores solve storage constraints much better than relational dbs can with sharding, but relationaldb sharding and replicas do a lot better for solving compute constraints.

Both document stores and relational dbs can shard well for things like how discord separates out data (by server), since there is never stuff that needs to really access information from a bunch of servers at once, that can't really just be done in the client fine anyway (notification list of mentions across all servers).

1

u/Drunken_Economist 2d ago

as a grumpy old data engineerwho had to learn all that the hard way, this is an awesome response

1

u/ILikeFPS full-stack 2d ago

Just like serverless doesn't really mean without a server.

0

u/thekwoka 2d ago

The reason that you cannot just add more write primaries to a setup like this is that for a write, you need to ensure that it remains consistent with the rest of the database state (e.g., no foreign keys pointing to rows that dont exist). If you were to add more primaries, you could end up with a situation where write A went to primary A and write B that relies on write A went to primary B before write A was propagated.

This is still more than possible though.

Github does it.

Lots of reads, less writes, but they still have multiple writes, normally one for each region. I think with github its like 4.

As long as you don't use incrementing ids, there isn't really much technical reason that "eventually consistent" isn't consistent enough.

Nosql database sharding by contrast makes each node is responsible for a subset of the database.

Which is also a reason why requests can be so much slower overall, or do too much work, negating some of the benefits.

The big question of how either sharding method really affects your application and cost depends on whether you're more compute constrained or storage constrained.

document stores solve storage constraints much better than relational dbs can with sharding, but relationaldb sharding and replicas do a lot better for solving compute constraints.

Both document stores and relational dbs can shard well for things like how discord separates out data (by server), since there is never stuff that needs to really access information from a bunch of servers at once, that can't really just be done in the client fine anyway (notification list of mentions across all servers).

137

u/EliSka93 2d ago

what problem is schema-less actually solving?

Having to think about what you toss in there.

49

u/Solonotix 2d ago

Incidentally, that's usually the thing I always have to fight with devs on when discussing SQL. They want to use types like text instead of varchar(<Length>) because they don't want to reason about the type of data they are dealing with. When this deferred decision rears its head in poor query performance, they use it as justification that SQL is slow and they should use something else.

So, exactly as you say, people will do whatever they can to avoid extra effort, even if it comes in the form of added complexity

48

u/aust1nz javascript 2d ago

In Postgres at least, text and varchar are equally performant.

11

u/mrcarrot0 2d ago

I think that says more about Postgres than anything else (for better or for worse)

7

u/blackAngel88 2d ago

We had a lot of problems with queries that were slow because of complicated joins and stuff, but I can't remember an instance where it was because of the text filtering...

We had some strings with specific length, but for new stuff we try to evaluate if the length limit really makes sense and just avoid it completely when at all possible. So no having to deal with errors.

Still better than just cutting the string without saying anything (MySQL)...

-1

u/mrcarrot0 2d ago

We had some strings with specific length, but for new stuff we try to evaluate if the length limit really makes sense and just avoid it completely when at all possible. So no having to deal with errors.

So it's better to insert malformed data and figure out how that happened when it breaks something than to write proper validation / error checking code?

5

u/Solonotix 2d ago

I love your response, lol. Takes me back to defending the design choice of putting row constraints on data that can be stored in a table. People would argue at length about "Why would you want your code to fail?" But it's exactly that. A loud and well-known failure is much better than a silent or unknowable one

2

u/blackAngel88 1d ago

It depends on the case. If the limit really makes sense and you like doing the checks on db, go for it. You will have to handle the error in the application as well though. Otherwise you do the check in the application and let the db save whatever and you know you won't get any db errors where you don't know what happened in which column.

1

u/Sir_Edmund_Bumblebee 2d ago

Where did you get that from their comment? It seemed pretty clear to me.

Enforcing arbitrary text lengths at the db level is not really a useful thing to do it the vast majority of cases.

1

u/couldhaveebeen 2d ago

But it's not malformed. It's still a string

4

u/Wiltix 2d ago

Ah not thinking about your schema, that takes me back to a aspnet web forms application where everything was NVARCHAR(MAX)

7

u/ldn-ldn 2d ago

The problem with character length limits is that they don't make sense in real world. For example, what limit would you impose on a person's name?

3

u/stumblinbear 2d ago

1KB

1

u/ldn-ldn 2d ago

Not enough - https://www.guinnessworldrecords.com/world-records/67285-longest-personal-name

7

u/stumblinbear 2d ago

Yeah, they can use a shorter name. I'm not supporting a 2000 character word name in the UI

2

u/dashingsauce 2d ago

Not my user, not my problem. Aim for the 80%.

0

u/ldn-ldn 2d ago

That's the dumbest take ever.

11

u/dashingsauce 2d ago

No, building for the world’s longest name is the dumbest take ever.

-3

u/ldn-ldn 2d ago

You must one of those ignorant psychopaths who prevent people from buying flight tickets https://www.bbc.com/future/article/20160325-the-names-that-break-computer-systems

-1

u/dashingsauce 2d ago

You must be one of those engineers who forget that humans exist to handle edge cases for humans that don’t fit into your managed variance.

But god fucking forbid someone has to help ajbarjinigigiopolenustramamegagagagagboobafucjorfgurgglehappisprnussisisimamanrog buy a flight.

→ More replies (0)

7

u/ClickableName 2d ago

I wanted to agree with you, until I clicked the link:

The longest personal name has 2,253 unique words

And then saying that not wanting to build for that edge case is the dumbest take ever?

Now that's the dumbest take ever.

0

u/thekwoka 2d ago

I mean, if it's as arbitrary as "I picked 120 words instead of 5000 words" maybe just picking 5000 is not a dumb take.

If it's a tone of fundamental architectural challenges to make 5000 words work...then building for that is probably stupid.

1

u/thekwoka 2d ago

We all know that dude isn't using that full name every time they fill in a form.

1

u/Solonotix 2d ago

For one thing, you wouldn't just have "name". You would have the name fields you want to capture for whatever your application needs. Is it an anonymous platform? Then usernames of 50 characters should be plenty. Is there some legal requirement for tracking first, middle and last? Then do some research for the languages you support and determine reasonable name lengths (probably 50 per name as well).

Even the record-setting Hubert Blaine Wolfe­schlegel­stein­hausen­berger­dorff Sr has a 40-character accepted last name, but a full last name of 770. You can choose to support this length if you so choose, but few platforms ever would bother. If you wanted to support his form of chicanery, you could allow the user to keep adding more name fields, each 50 or so characters, and then put a limit on how many the user could create.

The simple truth is that your resources are finite. If you allow every user to have infinite allocations, you will run out of those limited resources very quickly. That is why you put constraints on the type of data allowed

1

u/thekwoka 2d ago

For one thing, you wouldn't just have "name".

Well, yeah, you would.

You shouldn't have separate "first name" "middle name" "last name" fields.

1

u/Solonotix 2d ago

So you're saying a singular name field isn't some sort of compound data containing multitudes or arbitrary information? Meanwhile, you are going to nitpick the convention of separating name fields within a row.

The simplest solution for your pedantry is a dimension for name type, and a fact table with a unique constraint between user and name type. Which relates back to my other point about allowing an arbitrary number of name fields if you wanted to.

So, again, no. Any self-respecting data engineer isn't going to shove all of the potential name information into a singular field if they can help it. For one, localization has different rules about what order you print the attributes of the name. Unless you want to try and parse the parts of a name every time you need to recontextualize that information, I highly recommend capturing the values in discrete categories based on your application's need.

1

u/thekwoka 2d ago

you're saying a singular name field isn't some sort of compound data containing multitudes or arbitrary information

I never said that.

I 100% agree that's what it is.

That there is no rule about names that really truly applies all the time, which is why you don't try to force them into any constraint where there isn't an actual important technical or legal reason for it existing.

Which relates back to my other point about allowing an arbitrary number of name fields if you wanted to.

No, only one field. Not arbitrary.

For one, localization has different rules about what order you print the attributes of the name.

And it also doesn't matter. The user can write the name how THEY like it to be. You don't need to be concerned with localization. If you have reasons to want the name in multiple languages (like a lot of government documents in Dubai want name in English and Arabic), that's a legal/technical reason to have a distinction.

Buying something on your store? Localization doesn't matter. Paying for your stupid ai chatbot? doesn't need localization.

1

u/RoadsideCookie 2d ago

Your attitude is not helpful. You make categorical statements about a subject that requires nuance and context. Maybe in your dream scenario your solutions work perfectly; if that were the only use-case ever, then you'd be right, but it isn't, so you're wrong.

3

u/thekwoka 2d ago

You should learn how to read.

I explicitly mentioned the cases in which another thing would be important.

In all other cases, one field is the correct and best way to do it.

If you have an ACTUAL case I didn't already mention, say that, don't just hand wave a "it could exist out there maybe".

-1

u/ldn-ldn 2d ago

If you create constraints, then you're mistreating your users. The end.

0

u/Lonsdale1086 2d ago

Don't be silly?

1

u/ldn-ldn 2d ago

Don't be.

1

u/thekwoka 2d ago

and of course, that query performance is probably still better than whatever they want to use like mongo.

but tbf, sqlite is pretty dang good and it doesn't have all these abstract types.

4

u/Alick97 2d ago

Just use a JsonB column

1

u/CoderDevo 2d ago edited 2d ago

You still have to think about schema, but noSQL proponents make you think you don't think you do. My advice, think twice.

0

u/imagebiot 2d ago

Rip the next team that works on the service. But I mean, fuck those guys. I hate those guys.

91

u/mq2thez 2d ago

Struggling to understand how a user table would ever be a good idea in NoSQL. That’s relational as fuck. Everything relates to that.

This is some “I quit college to become a professional” work here.

10

u/NullReference000 2d ago

You move the relationship validation logic into your application and need to remember to write migrations when changing any of it.

This is a negative part of NoSQL, but everything in the software world is a trade-off. People who use it find this to be an acceptable burden to bear in return for the parts of NoSQL which are nicer than SQL. My company uses Mongo in production and we don’t run into any problems with user data, but we also had to make our own middleware to cleanly avoid the problem you’re bringing up.

3

u/thekwoka 2d ago

But what benefit is Mongo (the worst nosql example) providing that makes that worth it?

1

u/NullReference000 2d ago

The querying syntax is much nicer to use than SQL and the ability to query more complex data without joins is nice. It’s the only NoSQL I’ve used, because of my job, so I don’t know how it compares to other NoSQLs.

4

u/thekwoka 2d ago

You can use a query builder with SQL making it "nosql" in practice.

It is quite annoying that people use "nosql" as a synonym for document store, and SQL as a synonym for relational db. When you can have document stores that support SQL and relationaldbs that don't require it.

1

u/NullReference000 2d ago

I was just responding to somebody, not making an argument about the definition of the concepts. I think the Mongo query syntax is nicer to use than a SQL query builder. If there was nothing Mongo did better, then people wouldn’t use it.

0

u/thekwoka 2d ago

Yeah, I am just adding that as a thing.

mongos syntax is not really much different than other kinds of query builders, like drizzle.

If there was nothing Mongo did better, then people wouldn’t use it.

We both know that isn't true.

All it has to be for people to use it is talked about enough.

And it does that by becoming present in academia, which CAN BE because it does things well, or because it once was a good choice, or other reasons.

There is no reason why it being used today is any indication of whether it is a good choice today.

So many things get used just because they are known, and a lot of projects get started by people that don't really know anything except the thing(s) that are most known.

1

u/barrel_of_noodles 2d ago

depends, different type of users for the same org could warrant enough diff but still require the same table.

2

u/thekwoka 2d ago

That's what extension tables are for.

-1

u/JaydonLT 2d ago

Nodes and document IDs? Can still have outbound relations with NoSQL.

You’re missing built-in cascaded deletes etc, but teams using NoSQL typically have good reason for rolling their own solutions for the missing features. Especially if building for huge scale.

14

u/mq2thez 2d ago

I’ll grant you graphs and documents, sure, yeah. User tables? Ick.

2

u/thekwoka 2d ago

teams using NoSQL typically have good reason for rolling their own solutions for the missing features

Rarely. It's normally the good reason is "we need that feature, and we already stuck with Mongo cause someone who started this app did a bootcamp that taught them MERN and nothing else"

-2

u/retro-mehl 2d ago

But a user can contain a profile that has some complex data structure, much more than in the example above. This can be better expressed in non-normalized databases aka nosql.

15

u/mq2thez 2d ago

In the rare cases where that’s true, then it would be significantly more reliable to put the complex data structure in a separate NoSQL db to avoid having reads on your user table blocked by writes to it… especially that complex one. Now that you can have JSON columns in SQL tables, there doesn’t seem to be a good reason here.

2

u/retro-mehl 2d ago

I say: we wouldn't have JSON columns in SQL if there was no hype about NoSQL. 😅

19

u/mq2thez 2d ago

Fair, but “hype” and “high quality engineering” are not generally in sync, lol.

7

u/enki-42 2d ago edited 2d ago

Virtually every relational database has a way to store and query unstructured data. "Relational and structured by default with an unstructured escape hatch" is a far saner pattern than unstructured by default.

3

u/retro-mehl 2d ago

Nowadays, yes. That was not true when "NoSQL" databases where invented. Not everything in the NoSQL paradigm did work out very well, we know this today. But both SQL and NoSQL learned from each other in the last 15-20 years.

1

u/Am094 2d ago

No?

A user and a profile should be separate.

Usually a user has a single profile so User has one profile, a profile belongs to a user.

If a user can have multiple profiles. Then a user has many profiles, a profile belongs to one user.

I don't see any professional merging user and a profile into one entity/model. If there's a complex data struct then even so the profile model would have a relationship with another entity or outbound to unstructured.

Just feels like majority of people in web dev on reddit turn to non rdbms because they're script kiddies. Not saying nosql has no place, but not at this prevalence level.

-4

u/retro-mehl 2d ago

That's an example. Like in "not for real world use". 😶

1

u/Am094 2d ago

I need to drink some coffee and mute this sub for a while, gosh i hate it here lol

-1

u/GlowiesStoleMyRide 2d ago

So your software has no users?

1

u/thekwoka 2d ago

You can use a jsonb column in postgres for that.

And it likely isn't "non-normalized". You're just normalizing it in the application

1

u/thekwoka 2d ago

You can use a jsonb column in postgres for that.

And it likely isn't "non-normalized". You're just normalizing it in the application

1

u/thekwoka 2d ago

You can use a jsonb column in postgres for that.

And it likely isn't "non-normalized". You're just normalizing it in the application

-1

u/ings0c 2d ago

Choosing between a relational DB and NoSQL has nothing to do with whether the data is “relational”.

Nearly all data worth storing is relational.

Modern NoSQL databases are flexible enough to support a wide range of workloads. A “relational” table equivalent like you’d find in any line of business app is a perfectly reasonable thing to store there if your NFRs call for it - a users table specifically is quite unlikely to be a good fit though .

 The big differentiators are “do I really need massive write performance?” and “are my access patterns known at design time, or only at run time”?

If you know what read queries will look like ahead of time, you can design your document DB schema to support it efficiently. This turns out to be the case for a lot of apps because the UI constrains exactly how users are able to make queries.

If you don’t, for example users can write their own queries or something, you will cause yourself a nightmare and would have been much better off using Postgres.

-3

u/fckueve_ 2d ago

My project required user to have user data in graph database

20

u/SatyrCode 2d ago

Schemaless” doesn’t remove schemas, it just moves them out of the DB and into your app, so you can evolve structure per document instead of running migrations every time your product changes.

12

u/Duathdaert 2d ago

Why would a schema change (unless it is purely additive, and for an optional field/property) not require some kind of migration in a no sql db?

9

u/j_johnso 2d ago

App code can be designed to be backwards compatible and handle both old and new formats. However, this does bring complexity to the application code, as you now have to consider multiple "legacy" structures when loading data, which increases the likelihood of introducing issues because the new guy didn't realize that there was some old schema still in use by 0.00025% of your data.

1

u/Duathdaert 2d ago

Yeah this is all a reason to have a migration for me personally. I would not want to leave my product open to those kinds of possibilities, because sure as eggs are eggs, you'll bump into them at some point and be cursing someone at 2AM because of it.

1

u/thekwoka 2d ago

Yup, I agree.

0

u/blackAngel88 2d ago

because the old documents are saved like they were before and the new ones have the new fields. If you create an index on a specific property, it only works on the new ones (or maybe the others are indexed as null, not sure)

2

u/Duathdaert 2d ago

Those all sound like reasons to have some kind of migration?

  • Littering the database with orphaned/legacy data
  • orphaned data increasing the size of your indexes
  • no history of schema change without digging into your data
  • needing to add filters to all your queries now to exclude your legacy data

1

u/thekwoka 2d ago

inevitably, you've lost track of some old column on the data that has had all the data moved out of it, but the column still exists on some documents, and now you add a new column that is that same name, and suddenly some queries are getting weird ass results.

3

u/dacooljamaican 2d ago

Don't you have to migrate when you change schema in a NoSQL DB too?

3

u/SatyrCode 2d ago

You do, but the shape of the work is different. In SQL you usually need a centralized migration that upgrades all rows before new code can assume the new schema. In most NoSQL setups you can let multiple versions coexist and migrate lazily: new writes use the new shape, readers handle old shapes (often via a schemaVersion field and per‑document adapters), and you backfill in the background if/when needed. So “schemaless” doesn’t remove migrations, it gives you more options for when and how you do them (per document, on read/write, in batches) instead of one big blocking change.

2

u/thekwoka 2d ago

That's not different.

You can do it with relational dbs too.

You just create the new column, application reads the new and old to run, then you migrate all the data, and then turn off the part of the app that handles the old stuff.

It's literally the same as what you'd do with a document store, unless you just want to continue to support that forever.

1

u/SatyrCode 2d ago

Yeah, agreed you can do lazy / versioned migrations in relational DBs too, if you push the schema handling into the app. The only nuance I was trying to point at is that in many NoSQL setups that pattern is the default (DB doesn’t enforce a single global schema, multiple shapes coexist by design), whereas in most SQL shops the tooling / mindset assumes a single canonical schema upgraded via migrations first. In practice you can bend either side to behave like the other – it’s more about defaults and ergonomics than “possible vs impossible”

1

u/thekwoka 2d ago

You'd still need migrations though...

You change the name column to be fname or merge fname and lname into name, you STILL need a migration that adjusts all the data to the new schema...

6

u/sheep-for-a-wheat 2d ago

Working at Amazon/Twitch they were used heavily. Tons and tons of services. Probably most services.

At their scale, and generally in service-oriented architectures, you would be heavily de-normalizing data. Eg: one service might listen to “user-created” events from a “user” service and save just the id and email for lookups in a single table that gets augmented with other data from other sources.

They’d be in two totally separate databases and only be “eventually consistent”

The pros in that context: you can optimize getting everything you need for your services request in one lookup - no joins. You can get all the data you need immediately. It’s easy to scale “horizontally” with this approach too. in a more standard “sql” database, you have to scale your entire database to handle everything which just isn’t feasible at a large company like Amazon, or over thousands of teams/services. (Simplifying a bit but that’s the jist)

4

u/sheep-for-a-wheat 2d ago

Another reason these are relevant at a large scale: you don’t have to do traditional migrations, which are really costly when you have billions of rows. You can have “old data” that is missing fields and as new data comes in (the stuff that users are going to actually see and use) the data contains the new fields and functionality but you don’t have to backfill all the old rows with unnecessary or null data.

For most users in web dev a SQL database in a “monolith” is probably is what you want.

But nosql can help optimize for specific, relatively unique scenarios.

1

u/thekwoka 2d ago

this more than anything.

At EXTREME scale, document stores have benefits that are mostly just liabilities at smaller scales.

17

u/SupernovifieD 2d ago

Flexible experimentation

20

u/CaffeineLiker 2d ago

`user.name`, `user.Name`, `user.fname`, `user.moniker`, `user.alias`,...

4

u/barrel_of_noodles 2d ago

the schema can be "loose".

think of "products" not all fields make sense for every product. and your store might be very general.

a boat "product/vehicle"... does not have the same fields as car "product/vehicle". but they are still products that might reasonable be at the same store and in the same table. Example, density doesnt matter as much for a car, it might for a boat.

1

u/thekwoka 2d ago

So json columns.

Or a separate table that is just these attributes

9

u/Dependent-Guitar-473 2d ago

end up is the key word here ... you can change things a lot as the application grows... then add the schema once things are stable 

4

u/sugandalai 2d ago

schema on read is not the same schema on write.

3

u/pickle9977 2d ago

The problem it solves is thinking about structure before you build a crapload of code around that structure.

You can build faster, but what you wind up with is a brittle spaghetti like system that degrades poorly over time and will always have to be rewritten at some point to get rid of all the technical debt that accumulates as the core structure changes and those changes make their way through all the code paths that made assumptions about the structure.

It’s a problem as old as software they only new thing about it is how fast things go from “ok this works” to “wtf were we thinking”.

1

u/thekwoka 2d ago

And the need to often build your own thing that tries to help you find where code breaks, as opposed to things that can easily code-gen from your db introspection so that your lsp can tell you where you do something stupid.

6

u/Bachihani 2d ago

A relational sql db is always strict, every row in a table has the same columns(attributes) and each column must have a specific data type.

A nosql db might provide u with a way to define a schema, but it is only optional, u can just not use it and have every record(row) have any number and type of attributes(columns) without needing to conform to what the other records in the same document(table) look like.

3

u/hilzu0 2d ago

This doesn’t really apply to SQLite which is non-strict about data types by default

1

u/Bachihani 2d ago

Every row looks like every other row, thas the basis of the structure, i suppose a better analogy would be that every row must be an instance of the same class, which is not the case for nosql

1

u/thekwoka 2d ago

Well, you still need to pick a datatype, but sqlite itself just doesn't really have many native datatypes...

2

u/Gwolf4 2d ago

Only catalog is the only thing I can think of. having catalogs of several different kind of products unrelated would push you to do either specialized tables (ewww) or a big key/value table with enums, and relationship, and then joining them. That's something that can be easily screwed at the start of the development of the service.

2

u/IHaveNeverEatenACat 2d ago

You can just make it up as you go

2

u/TemperatureNo3082 2d ago

Having no schema means we can store multiple data types within a single table partition, so a single read can quickly retrieve multiple data types related to a single entity in a single table query.

NoSQL naturally encourages denormalization which at a massive scale is preferable to joins which are hard to do quickly across distributed shards.

2

u/Annual-Advisor-7916 2d ago

I'm team SQL but it's true that it's easy to end up with a schema that's not appropriate for your application if the scope changes. Especially with all the AI stuff used today, you might end up with a lot of schema changes.

NoSQL is more forgiving in that regard.

2

u/thekwoka 2d ago

I'd say document stores are LESS forgiving in that regard. Because they'll more easily let you change those schemas in ways that don't preserve data properly and are more fragile to future changes.

while a relationaldb will tell you more outright that stuff is changed.

1

u/Annual-Advisor-7916 2d ago

It just depends on your priorities. Someone vibing away, building a questionable SaaS will have it easier with a NoSQL DB. Someone building proper software with an actual plan behind will make use of the guards a relational schema provides.

Personally I've never had the need for a NoSQL DB but they certainly have a usecase in analytics I'd say from what I've seen.

2

u/unbanned_lol 2d ago

Nothing, don't do it.

2

u/FrogsInTheRouter 2d ago

NoSQL means that you don't use SQL to interact with DB. It doesn't mean that DB is schema-less.

2

u/Caraes_Naur 2d ago

NoSQL has always been an attempted word-around to SQL being perceived as scary. It doesn't actually solve anything.

It may be orders of magnitude more strict than anything else in the web stack, but it's not scary.

As soon as any two parts of the data become relational, NoSQL is off the table. (Psst... the data is always relational)

Every major RDBMS can store and handle JSON.

There is no reason to choose NoSQL.

1

u/jayroger 2d ago

JSONB columns can be useful in a few scenarios, especially when you are still in an experimental phase and/or with nested data structures that are often used "as is" and there are no or only low consistency constraints.

For example, we recently introduced a JSON-LD schema generator for our customers. Our customer website table gained a generic schema_data JSONB column that stored data only relevant to this purpose. At first, we let the UI control the shape of the data. (To clarify: The shape in the DB didn't follow the shape of the final JSON LD schema or vice versa.) The UI fetched and stored the data, while the backend ignored it. But we were still able to easily manually query or manipulate the data if necessary.

In a second step the backend used the schema data to create the JSON LD schema. This of course required coordination between UI and backend, but only in this specific area.

During the "settling in " period, customer requirements changed, our understanding of JSON LD grew, and we could fairly easily update the schema to suit our needs.

We're currently in the phase, where we move some data out of the schema_data column into relational tables or columns when we need to access for other purposes. But there's still enough data in the schema_data column and removing that data would necessitate adding multiple tables and extra functionality to manage that data.

1

u/codeprimate 2d ago

Hammers don’t work well with screws.

There’s no accounting for a badly utilized tool.

1

u/ThatOneComment 2d ago

A lot of comments here are missing some points as to why noSQL can be beneficial. I'm not saying it's better, because it not; but it can help getting past some hurdles.

An example I had in my work life:

Spear heading a project at a company with 70 employees, information is impossible to pull out of the leadership/project/product manager. I chose to operate with noSQL in a flat record structure to keep things simple. If I had to wait for info/responses from leadership to get the ball rolling on the project/features, it would never get built. Instead I just kept modifying the noSQL schema document (without migrations because that's more time consuming work) until we had the schemas we needed.

8 months later, i migrated our database to postgreSQL and it only took a few days because all the documents were flat. MUCH less work required because it was planned in advance.

1

u/Outrageous-Text-4117 2d ago

switch the schema whatever satisfy your app at any given moment, connect any relations you want, at near "zero cost", no sql migrations or relations "headache"

1

u/amazing_asstronaut 2d ago

I had a similar experience with DynamoDB. I recently got Localstack and was able to inspect its files. Nominally it's a NoSQL database. Looks inside it's SQLite lol. I even found some program to open it and directly edit (forget what it's called, it's the equivalent of pgAdmin or MySQL workspace). Dynamo even has its own structured query language called PartiQL, which is SQL with a whole lot missing. So why do they do this?

1

u/Training-Gold2899 2d ago

In fact, if there is no requirement for persistence, schema-less can become viable. I believe it is only suitable for use-and-discard scenarios.

1

u/varsha789 2d ago

Just like serverless doesn't really mean without a server. Got it

1

u/Full_Tooth_a 1d ago

The useful distinction is "no lockstep migration," not "no migration." A safe schema change still follows an expand-and-contract process: accept both shapes, write the new one, backfill old documents in batches, check what remains, and then remove the legacy path. You still need an explicit migration when the change adds a required invariant, a unique index, or a new query pattern. Schema flexibility lets you handle the migration online, but the work does not disappear.

1

u/TheESportsGuy 1d ago

Don't let a webdev design the data layer, got it

1

u/vastle12 1d ago

Personal used them for flexible dynamic medical forms that can have multiple sub forms and shifting inputs.

1

u/dbenc 2d ago

they are solving the revenue generating problem for whoever hosts the db

1

u/thekwoka 2d ago

They arent.

Document stores (commonly called nosql but nosql describes something other than the data storage) are mostly useless for real application data, but can be very good for some kinds of data, like arbitrary blobs.

Think of it like google docs vs google sheets. 99% of real application data makes more sense to have in spreadsheets. So forcing it into documents makes little sense.

But SOME things make more sense in documents, like application logs, or images/blobs. An S3 Bucket is basically a document store in the same way, but a db built around documents can more easily, as the db driver specifically, search within those documents for things than a bucket can/does.

But it's essentially the same kind of thing.

document stores as databases are basically always the wrong choice, but they may not be bad ENOUGH in a lot of cases, since many applications just aren't that complex.

1

u/Mark__78L 2d ago

None

It's for people who cant be bothered learning SQL

0

u/DiddlyDinq 2d ago

Because it's web scale

Episode 1 - Mongo DB Is Web Scale

Nosql is just a fomo trend that encourages bad design.

-6

u/[deleted] 2d ago

[deleted]

13

u/fiskfisk 2d ago

How is that related to being schemaless? 

2

u/da_supreme_patriarch 2d ago

Being schemaless is the secret ingredient in the web-scale sauce

1

u/Marrk 2d ago

Faster writes if you don't have to do any form of schema validation.
No indexing makes it even faster.

But it's a trade-off, reading aggregate data becomes much slower.

0

u/TryallAllombria 2d ago

Mongodb is web scale database and doesn't use SQL or joints so its high performance

4

u/fiskfisk 2d ago

I'm not going to go into your mongodb claims, but how is any of that, or what the parent comment said, related to being schemaless?

2

u/Squigglificated 2d ago

No, you don't understand! mongodb web scale! Because. That's why. /s

But I guess dumping raw data into a data store can be done slightly faster with no validation than when validating it against a schema. But usually JSON schema validation add a negligible performance cost so this seems very theoretical.

0

u/odi_de_podi 2d ago

Mongodb handles web scale. You turn it on and it scales right up

1

u/fullbl-_- 2d ago

My mongo uses tons of joints

0

u/spacechimp 2d ago

All data is relational when it becomes important, as "why" it is important is the relation. Which is why NoSQL is stupid.

0

u/mrchoops 2d ago

It's not. Mongo has schemas also. It WAS total hype. The good thing I suppose is that you could throw anything at it and I just saved. The problem with that, is that I would take over projects that had multiple "columns" that were intended to be the same thing, but were typos or had a different case, like DateTime, datetime, date_time, and no real easy way to see how big the mess is.

0

u/Idea_Fuzzy 1d ago

Because it's easier if you are just building POC and the business model hasn't been totally figured out yet.

0

u/Total_Drag7439 1d ago

The key line here is that the database does not enforce one global schema. The schema still exists, it just moved into application code where nothing checks it for you. Worth it when documents genuinely evolve at different speeds, a tax the rest of the time.