_______ __ _______
| | |.---.-..----.| |--..-----..----. | | |.-----..--.--.--..-----.
| || _ || __|| < | -__|| _| | || -__|| | | ||__ --|
|___|___||___._||____||__|__||_____||__| |__|____||_____||________||_____|
on Gopher (inofficial)
URI Visit Hacker News on the Web
COMMENT PAGE FOR:
URI SQLiteData: A fast, lightweight replacement for SwiftData using SQL and CloudKit
jmull wrote 4 hours 29 min ago:
The abstractions you build your code on top of can be very costly, and
a lot of that cost is due later.
You need to carefully weight the costs of abstractions you adopt. That
goes double (or triple or quadruple) for data access, which is
typically the central point of an app.
When sqlite is involved, I'm really doubting an ORM is worth it in the
great majority of cases.
That's because sqlite is robust, mature, full-featured, very well
documented, has demonstrated long-term viability, has a commitment to
backwards compatibility and support, etc. I think it makes the most
sense to implement an application-level data-access layer and have it
use sqlite as directly as possible.
What sqlite doesn't do that an app needs is dealing with Swift types.
I.e., the grunt work of the sqlite3_bind_* and sqlite3_column_* calls
to transform Swift values to and from sqlite3 values. (And to a lesser
extent, various Swift-isms for quality of life.) But you can have that
without the rather more intrusive footprint -- and therefore much
higher cost -- of an ORM.
stephencelis wrote 3 hours 51 min ago:
Are you suggesting SQLiteData is an ORM, or SwiftData? SwiftData (and
CoreData) are certainly ORMs, but SQLiteData is not. It's simply a
collection of tools on top of SQLite that provide similar
functionality to SwiftData, but you always have direct access to
SQLite.
The things you say that SQLite doesn't do is exactly what SQLiteData
provides (Swift-friendly bindings for encoding and decoding data from
SQLite), and more. There's a footprint, as there is with any library,
but there is no ORM level of abstraction here.
jmull wrote 1 hour 28 min ago:
Well, it has things like @Table and wants you to write SQL using
Swift syntax. It's good that it lets you bypass this, but it
encourages using the wrappers so I think that's going to be most of
the usage. Also, if you're going to mostly bypass, then why incur
the costs of adopting? The main bypass I saw was #sql, but while
you can write SQL, it's a substantially different API than sqlite's
C API to do the same.
One problem with data access wrappers is that what's both above it
and below it have strong app concerns. The developer needs to
understand and control what's below it, so an intermediate
abstraction gets in the way. That is, in addition to understanding
the lower-level, they also have to understand the intermediate
abstraction, and how it maps to the lower level. So it's really
best if the API surface is minimal.
What I would want is an API where the core is something like:
bindParameters(pStmt, anEncodableThing)
readRowColumns(pStmt, ADecodableThing.Type) -> ADecodableThing
where pStmt would be as minimal a wrapper around a sqlite statement
as is feasible (maybe even a pointer to a sqlite3_stmt directly?).
There might be a minimal (non-existent?) wrapper around the sqlite3
connection too. (The sqlite pointer should be a public member of
any wrapper, so you can call any sqlite3_ functions you want).
(I'd want some convenience methods too, that combine bindParameters
and readRowColumns with preparing a statement and stepping through
it, like sqlite's exec.)
Now, I know this doesn't address CloudKit sync at all, but I think
a similarity minimal type-focused binding is best there too. It has
nowhere near the 5-star API that sqlite has so there's a better
argument for wrapping it, but sync tends to quickly accumulate
app-level concerns when it comes to the exact details of sync
conflicts, so you might as well keep that at app level. I think
there maybe a set of composable convenience methods might do it, to
handle the pain points without hiding the details you'll need
access to.
Anyway, my point is NOT that sqlitedata is bad, it's that people
should be really careful about taking on the costs that it has, and
consider whether it will ultimately cause more problems than it
solves. Meanwhile, sqlite is world-class -- you want to wrap it as
little as possible.
wahnfrieden wrote 1 hour 17 min ago:
World-class doesnât help me if there is no off the shelf or
easily made solution to CloudKit sync. Which hasnât existed
until SQLiteData.
(Previously I rolled my own sync with RealmSwift but thatâs now
dead.)
Without that, my only alternatives are SwiftData or CoreData,
with severe downsides.
wahnfrieden wrote 4 hours 25 min ago:
How else will you support CloudKit with SQLite? Personally CloudKit
is a requirement as my customers value having their personal data
inside their own Apple account rather than storing it on my servers.
I also rely on CloudKit to avoid the cost and on-call burden of
operating a realtime sync service.
Re: ORM, the SwiftData-like ORM part of SQLiteData is actually
completely optional. You can write SQL statements directly via #sql:
[1] You can wrap usage of this in your own method so that you can
swap out SQLiteData in the future without lock-in.
Re: Swift types, you may roll your own mapping, but performance is
not trivial. Here is a benchmark from SQLiteData:
Orders.fetchAll setup rampup duration
SQLite (generated by Enlighter 1.4.10) 0 0.144 7.183
Lighter (1.4.10) 0 0.164 8.059
SQLiteData (1.0.0) 0 0.172 8.511
GRDB (7.4.1, manual decoding) 0 0.376 18.819
SQLite.swift (0.15.3, manual decoding) 0 0.564 27.994
SQLite.swift (0.15.3, Codable) 0 0.863 43.261
GRDB (7.4.1, Codable) 0.002 1.07 53.326
I found Enlighter and Lighter more intrusive to adopt, and the other
open source solutions far slower (too slow for my needs, where my
users have hundreds of thousands or millions of rows within the iOS
apps)
URI [1]: https://swiftpackageindex.com/pointfreeco/swift-structured-q...
mikeocool wrote 5 hours 46 min ago:
GRDB/GRDB-Query ( [1] ) is another solid library worth looking at if
you're looking for libraries in this space, though it doesn't have
CloudKit support out of the box.
The more SwiftData alternatives the better -- as it has a lot of rough
edges, and Apple hasn't invested much in it since it's initial launch.
URI [1]: https://github.com/groue/GRDB.swift
wahnfrieden wrote 4 hours 31 min ago:
> though it doesn't have CloudKit support out of the box
There is no CloudKit support for GRDB available anywhere except for
via this SQLiteData package
"Harmony" was an experiment in adding it to GRDB on iOS 17+ but
reports claim that it's broken. SQLiteChangesetSync is another
earlier experiment that is unmaintained
groue wrote 5 hours 16 min ago:
Thank you (GRDB author here).
It is not mentioned in the README of the repository, but SQLiteData
wraps GRDB to access the database and get notified of database
changes (the meat and butter).
GRDB is by itself a solid "toolkit for SQLite databases, with a focus
on application development", with both high levels APIs for everyday
coding, and expert SQLite features for the demanding developers. Many
apps rely on GRDB alone.
yAak wrote 2 hours 19 min ago:
GRDB is an invaluable tool to me and, IMO, to the Swift community
â thank you for open-sourcing your countless hours of work and
expertise!!
dgllghr wrote 2 hours 58 min ago:
Thank you for GRDB! I am using it in a project now and itâs been
great. About the benchmarks in this repo though, how can SQLiteData
be faster if it uses GRDB under the hood? Are they doing something
inefficient or are they bypassing GRDB in some way?
wahnfrieden wrote 2 hours 44 min ago:
GRDB appears to encourage Codable which is very slow. It does not
require it though and there are some alternatives, some of which
are also slow. ("Slow" can still mean "fast enough", depending on
the user.)
SQLiteData uses this library which uses Swift macros to generate
performant interfaces to schema statically at build time: [1] The
alternative I've seen for doing this with GRDB seemed more
cumbersome and lacks community adoption: [2] You must define
schema and queries in an unusual external file that a Rust tool
transforms for you.
There is also this library which does not use GRDB but takes a
similar approach to SQLiteData though you have to run a program
that generates the bindings outside of your normal build:
URI [1]: https://github.com/pointfreeco/swift-structured-queries
URI [2]: https://github.com/Jasperav/GRDB-ORM
URI [3]: https://lighter-swift.github.io/documentation/lighter/pe...
groue wrote 2 hours 16 min ago:
Yes. GRDB encourages Codable because the user can profit from
the code generated by the compiler, and this implies that
database values are accessed by column name, on top of the
Codable runtime, and those layers have a high cost. When
necessary it is possible to access database values by position,
and in this case GRDB achieves speed of light (performance
nearly identical as raw SQLite).
wahnfrieden wrote 2 hours 8 min ago:
From my understanding this is a sample of the database values
by position approach: [1] That approach benchmarks at 2.2x
the duration of StructuredQueries (45% as fast): [2] 18.819s
vs 8.511s
So it appears that there is lightning-fast and
lighting-faster.
Of course aside from comparing the dev ergonomics (138 vs 33
lines for the respective benchmarks), either may be fast
enough depending on the use case.
BTW I did also see some discussion in swift-evolution about a
future replacement for Codable but haven't tracked its
progress. I hope they do because Codable is very convenient
but tragically slow.
URI [1]: https://github.com/Lighter-swift/PerformanceTestSuit...
URI [2]: https://github.com/Lighter-swift/PerformanceTestSuit...
dgllghr wrote 2 hours 25 min ago:
Now I understand. Thanks!
mbw234 wrote 3 hours 38 min ago:
Sorry about that, just opened a PR to update the readme and docs
landing page! Previously it was very clear that GRDB was being used
under the hood, but now we will say it explicitly.
jparishy wrote 4 hours 36 min ago:
Used GRDB many times in a previous life, thank you very much for
your work
mikeocool wrote 5 hours 3 min ago:
Oh hi -- Thanks for your work! Just finished replacing SwiftData
in an app with GRDB and it was a pleasure to use.
After struggling with some issues in SwiftData, GRDB really hits
the nail on the head in terms of providing a solid dev experience
for the common cases, but allowing you to drop into the more
advanced features when you need them.
groue wrote 4 hours 44 min ago:
You're welcome :)
asdev wrote 5 hours 46 min ago:
Unrelated, but as a backend dev who recently worked in Swift/SwiftUI to
build an app, the iOS developer experience is absolutely horrendous
rTX5CMRXIfFG wrote 5 hours 28 min ago:
Heh, letâs see how you like Android and React
asdev wrote 5 hours 26 min ago:
React Native is actually quite good at this point, I've worked with
it pretty extensively before doing a native app. I prefer it so I
don't have to deal with iOS stuff directly
wahnfrieden wrote 4 hours 45 min ago:
What would you use from RN to achieve CloudKit sync? I haven't
seen anything notable.
Many users value iCloud sync because it keeps their data private
to their own Apple account, without having to share access to
their data with the app developer's servers.
rTX5CMRXIfFG wrote 5 hours 51 min ago:
I find "Fast lightweight replacement for SwiftData that can target all
the way back to iOS 13" (when SwiftUI came out I think) to be such a
hard sell. How many businesses are there that must absolutely (1) run
their apps on iOS 13; (2) use declarative syntax for data persistence;
and (3) are profitable enough to invest time and effort on learning a
third-party library for a really old OS? It sounds like an overly
specific use case.
In general though, I'm not sure who finds PointFree's work a net
positive investment of resources, but the company seems to be in the
business of reinventing the wheel and locking you in so that you'll pay
for support. Meanwhile, Apple's own SDKs are free. And with Apple's
history of source-breaking changes over major platform updates, plus
given how even huge libraries/tools like Alamofire, Realm, RxSwift,
Cocoapods eventually succumbed to oblivion, I can't think of why an
Apple developer with any modicum of discernment would choose
PointFree's tools over Apple's own--unless they are themselves caught
by the allure of reinventing the wheel.
mbw234 wrote 3 hours 51 min ago:
I'm half of Point-Free and can answer your questions.
The fact that our library deploys back to iOS 13 is just our way of
showing we care deeply about back deploying these tools so that
anyone can use them. Certainly no one is deploying iOS 13 apps these
days, but people definitely are deploying iOS 16 apps (and may be for
another year or two), which has no access to SwiftData.
And investing time into learning a third party library is just the
name of the game when one feels that the tools Apple provides do not
suit their needs. Luckily our library mimics many of the patterns
that one is used to with SwiftUI and SwiftData, but has the benefits
of being based on SQLite, is open source, and not a proprietary
technology.
And I'm not sure what wheel we are reinventing here. We feel most of
our libraries are specifically filling holes that Apple has left
open. We also don't do a ton of paid support with companies, but we
certainly do answer dozens of questions on Slack, Twitter, GitHub
discussion, etc, every day. All for free.
rTX5CMRXIfFG wrote 42 min ago:
This library is basically Swift/Core Data, just portable farther
back in time. Itâs not hard to point out where the reinventing
the wheel is.
> learning a third party library is just the name of the game when
one feels that the tools Apple provides do not suit their needs
Or you can just wait a couple more platform updates instead of
always being on the bleeding edge. Appleâs frameworks might not
be perfect, but theyâre quite exhaustive, and libraries that aim
to fill in those gaps are offering diminishing marginal returns
that would be unnecessary in 1-2 WWDCs.
Iâm sure there are apps small enough to plan for short time
horizons, but for companies who plan for their apps to run for half
a decade or more, this is the sort of library that youâll soon
mark as tech debt and then refactor for removal when its approaches
eventually make it to Appleâs own SDKs. The historical precedent
is there.
> open source, and not a proprietary technology
Yea those are great virtues elsewhere but they stop becoming
priorities the moment you pay for the license to develop for a
walled garden and agree to its T&Cs
wahnfrieden wrote 10 min ago:
> Or you can just wait a couple more platform updates instead of
always being on the bleeding edge.
So your advice is to use CoreData until 2028 or so until we can
terminate iOS 17 users and start using WWDC24 SwiftData.
And then maybe a few years from now (meaning usable by 2032)
Apple fills SwiftData's gaps that SQLiteData meets today (ability
to use it in background fetch tasks, scalable performance, usage
outside of SwiftUI views, fast JSONB storage & querying, FTS) or
has the capability to meet in the near future (sharing, public
database etc.).
That's neither compelling nor industry standard, and is why so
many businesses haven't adopted SwiftData and have moved on from
CoreData.
stephencelis wrote 3 hours 59 min ago:
I think you're missing the point. iOS 13 is not the value proposition
of the library, it's simply one small feature of comparison of many
and isn't highlighted beyond a simple mention. The library provides
just as much value if you are starting a new app today and choose to
target iOS 26+.
> And with Apple's history of source-breaking changes over major
platform updates, plus given how even huge libraries/tools like
Alamofire, Realm, RxSwift, Cocoapods eventually succumbed to
oblivion, I can't think of why an Apple developer with any modicum of
discernment would choose PointFree's tools over Apple's own--unless
they are themselves caught by the allure of reinventing the wheel.
Isn't this just a blackpilled take in general? You're complaining
that Apple software breaks, that other third party software is
discontinued, and this somehow leads to the conclusion to avoid this
library?
rTX5CMRXIfFG wrote 25 min ago:
If you think that through, the answer is: of course! Take for
example the move from Swift 1 to 2 (an extreme example but
illustrates the point).
If you used a third-party lib written in Swift 1 and you had to
move to Swift 2 because the next version of iOS requires the
latter. Then youâll have to wait for the lib developer to publish
a version of their third-party lib for Swift 2 before you can
publish your app. Thatâs the same kind of risk that youâre
exposed to with source-breaking changes.
Admittedly, source-breaking changes have gotten less frequent in
Appleâs major tooling updates, but the right mindset when
developing for walled gardens like Apple is that it will happen
again.
wahnfrieden wrote 5 hours 19 min ago:
The purpose is not iOS 13 (they perhaps should not place that in the
lede). But even iOS 17 is hard to use swiftdata on, given indexes
were not even added until 18. And many deploy still to iOS 17 or even
16. I donât know why you see support for that as a suspicious
negative.
But for me the draw is:
- Can use this outside SwiftUI views, unlike SwiftData
- Can adopt future improvements without waiting on annual release
cycle + several more years until I can sunset older OS compatibility
- Performance: SwiftData has severe performance issues that are
manageable with SQLite
- Cross-platform: SQLiteData depends on GRDB which is perhaps close
to merging support for Android (and already supports Linux), at least
as an experimental trait: [1] By using it with [2] I can deploy the
same SwiftUI codebase to both iOS and Android. And with SwiftCrossUI
I can target Windows, Linux too while reusing the same
model/persistence layer code. Perhaps this will work in WASM too.
And lastly, many apps already choose to use SQLite. You can see
several commenters here who use GRDB in their projects. This library
adds CloudKit sync to that which has previously not existed outside
of broken experiments or proprietary in-house solutions. The CloudKit
capability is more relevant than a decision for whether to use
declarative syntax.
Now that RealmSwift is dead, there are to my knowledge zero other
alternatives to this new library and SwiftData/CoreData for a
CloudKit-synced DB.
Re: pointfreeco Iâm unfamiliar with their other work and have no
interest in TCA. This library has no dependency on TCA and no lock in
on the esoteric or paid parts of their ecosystem that I can see. Most
of what it does is provided by GRDB, which has some level of
community support and production adoption even from within Appleâs
own frameworks (Apple ships GRDB inside of iOS & macOS!). Your
comments there seem like fear-mongering and irrelevant to this
project. I suggest evaluating this independent of any perceived
baggage from their brandâs other work.
URI [1]: https://github.com/groue/GRDB.swift/pull/1708
URI [2]: https://skip.tools
scary-size wrote 5 hours 54 min ago:
Love it! Not an iOS dev, but have built a personal app that I want to
publish for macOS too. I built on top of SQLite, but didnât want to
tackle the synchronisation yet. That should solve it.
It employs a âlast edit winsâ strategy for conflict resolution,
which is fine in my case. Obviously could be too basic for
collaborative apps.
jitl wrote 5 hours 47 min ago:
Youâd be surprised; it depends on the scope of the last write wins
register. If you do it granularly attribute by attribute, last write
wins is fine for mostly-online mostly-realtime collaborative apps.
The exception is (rich) text where you really want CRDT/OT/similar
intention preserving merges of concurrent edits.
wahnfrieden wrote 4 hours 41 min ago:
SQLiteData does do it granularly by attribute
DIR <- back to front page