I don’t know, SQLite it’s something that makes sense in theory, but I think its easier for ops people to just use a proper database. If you need to move the database to a separate machine, limit permissions, etc. its just easier to do.
SQLite is great for local apps that need a structured way to store their data, but I’m not really comfortable using it for more than that.
The database in a state where it’s violating some assumptions I’m making and I need to manually intervene without taking down my application for example. I need to have an audit trail on the changes being made to the database and who made them. I need to create replicas to implement failover. I need to replicate my application on multiple machines and all the replicas need to have the same view of the data. I need to mitigate the possibility of data leaks if I have multiple tenants sharing a database.
I’m not saying that you’re wrong for using it. I’m just saying that it doesn’t work for everything.
I typically run postgres locally too (in docker), while there’s still technically network overhead there’s not much compared to a real network, plus you can easily move it to another machine without reworking your app to switch from SQLite to postgres.
Personally I never do permissions at the database layer anyway - it’s always done at the application layer.
I also never move the database to a separate server - that adds too much latency. If your database is local, and “hot” tables are cached in RAM, you can do several hundred thousand queries in a split second and having performance like that can drastically reduce complexity in your business logic (and therefore, drastically reduce bugs).
Regardless, I don’t see it as something that is the silver bullet that people make it out to be. Being able to introspect the production database, query it, and generally have a set of tools to properly manage your data as opposed to having everything in a file fully managed by your application is something useful for me that you lose with SQLite.
I’m not sure I understand your point? You can connect to and run queries/etc on the production database in SQLite.
I’m not really advocating for using SQLite by the way - I’ve only ever used it on smartphone apps myself where a full database wouldn’t even have enough RAM to run at all. I’m just pointing out that permissions isn’t a feature I’ve ever found useful.
For example, say I have an invoice table that is written to whenever a customer buys a product. Customers need to have write access to the table in the database. But I don’t want them to be able to write anything they want - there needs to be severe restrictions on what can be written, and those have to be done outside of the database.
Since you have to do some of your permissions outside the database, it’s more reliable to just do all of them there. Splitting things up with half your security in one place and half in another is asking for bugs.
The main reason I would avoid SQLite is the backup system, which essentially takes your whole database offline (for write access anyway) while the backup is running. That’s just not good enough once the database reaches a size where backups take more than a moment. But if you’re not storing much data, or not doing many writes, that’s a non-issue.
SQLite definitely has advantages. It’s often extremely fast for example. The lack of complex features removes performance bottlenecks all of the place and you can do millions of basic select queries per second in SQLite. Obviously not every query is that fast, but a lot of them are especially if you design your indexes/etc properly.
Definitely not a silver bullet, but I do think anyone who writes code that reads or writes data should be at least aware of the basic capabilities of SQLite. It’s free. It’s reliable. It runs literally on any platform (you can even run it client side in a webpage these days). So the only reason to avoid SQLite is if it’s the wrong tool for the job. And you can’t make that judgement call unless you have experience with it.
SQLite should be in every developer’s toolchain, even if you don’t have a use for it right now.
It’s useful for audit trails and the like, generally OS audit logs only tell you who accessed the machine not what they did on the production database. Things like that. Databases like postgres come with admin tooling in general that SQLite isn’t really meant for. As you said, backups as well are a problem.
I don’t know, SQLite it’s something that makes sense in theory, but I think its easier for ops people to just use a proper database. If you need to move the database to a separate machine, limit permissions, etc. its just easier to do.
SQLite is great for local apps that need a structured way to store their data, but I’m not really comfortable using it for more than that.
SQLite is a proper database.
For single-instance deployments, running SQLite means no overhead due to a network roundtrip, and things just work.
Proper = has actual admin tooling vs. just a file format.
What admin tooling do you need? You haven’t defined any problem requiring a solution.
The database in a state where it’s violating some assumptions I’m making and I need to manually intervene without taking down my application for example. I need to have an audit trail on the changes being made to the database and who made them. I need to create replicas to implement failover. I need to replicate my application on multiple machines and all the replicas need to have the same view of the data. I need to mitigate the possibility of data leaks if I have multiple tenants sharing a database.
I’m not saying that you’re wrong for using it. I’m just saying that it doesn’t work for everything.
I typically run postgres locally too (in docker), while there’s still technically network overhead there’s not much compared to a real network, plus you can easily move it to another machine without reworking your app to switch from SQLite to postgres.
Use the tool for the job. It is simple advice, but it will make your life so much easier.
I do use the tool for the job. I don’t understand your point.
I was just reiterating your point and agreeing with you, bud.
Sorry about that, it seems I unintentionally created a bit of controversy and am being a bit defensive.
Personally I never do permissions at the database layer anyway - it’s always done at the application layer.
I also never move the database to a separate server - that adds too much latency. If your database is local, and “hot” tables are cached in RAM, you can do several hundred thousand queries in a split second and having performance like that can drastically reduce complexity in your business logic (and therefore, drastically reduce bugs).
Regardless, I don’t see it as something that is the silver bullet that people make it out to be. Being able to introspect the production database, query it, and generally have a set of tools to properly manage your data as opposed to having everything in a file fully managed by your application is something useful for me that you lose with SQLite.
I’m not sure I understand your point? You can connect to and run queries/etc on the production database in SQLite.
I’m not really advocating for using SQLite by the way - I’ve only ever used it on smartphone apps myself where a full database wouldn’t even have enough RAM to run at all. I’m just pointing out that permissions isn’t a feature I’ve ever found useful.
For example, say I have an invoice table that is written to whenever a customer buys a product. Customers need to have write access to the table in the database. But I don’t want them to be able to write anything they want - there needs to be severe restrictions on what can be written, and those have to be done outside of the database.
Since you have to do some of your permissions outside the database, it’s more reliable to just do all of them there. Splitting things up with half your security in one place and half in another is asking for bugs.
The main reason I would avoid SQLite is the backup system, which essentially takes your whole database offline (for write access anyway) while the backup is running. That’s just not good enough once the database reaches a size where backups take more than a moment. But if you’re not storing much data, or not doing many writes, that’s a non-issue.
SQLite definitely has advantages. It’s often extremely fast for example. The lack of complex features removes performance bottlenecks all of the place and you can do millions of basic select queries per second in SQLite. Obviously not every query is that fast, but a lot of them are especially if you design your indexes/etc properly.
Definitely not a silver bullet, but I do think anyone who writes code that reads or writes data should be at least aware of the basic capabilities of SQLite. It’s free. It’s reliable. It runs literally on any platform (you can even run it client side in a webpage these days). So the only reason to avoid SQLite is if it’s the wrong tool for the job. And you can’t make that judgement call unless you have experience with it.
SQLite should be in every developer’s toolchain, even if you don’t have a use for it right now.
It’s useful for audit trails and the like, generally OS audit logs only tell you who accessed the machine not what they did on the production database. Things like that. Databases like postgres come with admin tooling in general that SQLite isn’t really meant for. As you said, backups as well are a problem.