GeezerVison

Nice @jlongster

What do you think @lovasoa? I'm excited about this PR on principle.

Hello everyone!
I've been wanting to review this PR for a long time, but never found the time to get to it.
I realize that sql.js is stagnating mainly because of me, and I would like to add a new maintainer. @jlongster, you have been doing really awesome work around this project. Do you think you would have the time and energy to be a maintainer?

@lovasoa Any luck getting a new maintainer or a chance to review this? Would be a great to get these changes in so absurd sql could be used without using a forked version of sql.js

@lovasoa could you please review it? This change will make a big move in using of sql.js for the web 🙏

Hey @lovasoa - is there any way I can help here?

I can do a cursory review but I'm not deeply familiar with the internals of sql.js yet -- I hope to be there eventually.

I'm interested in the future of sql.js as the developer and maintainer of https://aphrodite.sh/ (multi-platform / local-first ORM)

considering that we now have https://github.com/whatwg/fs then i think indexeddb is rather 👎

@jimmywarting any idea on when StorageManager.getDirectory() will become available on Chrome for Android?

This sounds extremely promising but I need mobile support. I couldn't find any mention of this in Chrome Platform Status or Issues - chromium.

StorageManager API: getDirectory | Can I use... Support tables for HTML5, CSS3, etc

1352738 - StorageManager.getDirectory on Chrome for Android - chromium

i don't know.
technically there isn't any mobile specific UI issues about picking files etc, whatwg fs is purely about the api and adding a sandboxed fs
The "File system access" on the other hand is for a mean to request access to a user selected file or folder and requires UI for prompting access. The android issue are here: https://crbug.com/1011535

safari is actively working on whatwg fs behind a expe flag, but it sucks pretty much right now. You can only create folders and empty files. you can't read or write data to/from files atm.

i haven't heard anything from FF yet.

Anyway. i just learned that navigator.storage.getDirectory() is just an alternative new api for webkitRequestFileSystem(TEMPORARY, 0, success, fail)
modifying files in webkitRequestFileSystem will make an effect on navigator.storage.getDirectory and also the other way around

the benefit of using navigator.storage.getDirectory in web worker are that you can have both a async and sync variant

1352738 - StorageManager.getDirectory on Chrome for Android - chromium

you probably want to close this as dupl of https://crbug.com/1011535

also have a look at https://github.com/WICG/file-system-access/issues

I considered closing it as a duplicate but then I realized that as the Origin Private File System requires no user interactions nor permission granting that it might be best to track it separately. I had not found that other crbug though. Thank you for all the links!

I've built SQLite filesystems with both Origin Private File System access handles and IndexedDB. In my benchmarks OPFS was somewhat slower. This may in part be because I used non-synchronous access handles for reading database files (and synchronous handles for writing) to allow concurrent read access from multiple database connections, but is also because atomic commit and optional durability settings provide some implementation advantages for IndexedDB.

That said, unless performance or browser support is the overriding priority, I would much rather write code for OPFS than IndexedDB.

The file system standard looks like, but as far as I can tell from reading around the web, Firefox has no plans to implement support. For the moment, anyway, IDB seems to be the most compatible backend for SQL.js. Is this PR held up on review because of the file system access standard?

Firefox has objections to the full File System Access API proposal, but has been more encouraging on the parts used for Origin Private File System, including the AccessHandle API (which is what Safari has implemented). Their implementation activity has stepped up in the last few weeks so it looks like they are actively working on at least trial support.

Well that's very encouraging!

Any news on this? https://caniuse.com/?search=opfs Apparently OPFS is avaliable on most browsers now and one intended use case is actually as storage for sqlite-wasm databases in web workers

Both IndexedDB and OPFS access handles can be used for persistent SQLite storage in the browser. At the moment it appears that neither is clearly better than the other for this purpose; it depends on your application requirements.

• IndexedDB
  • Lower write transaction overhead.
  • Allows concurrent reads.
  • Allows trading durability for performance.
• OPFS access handles
  • Faster raw reads and writes.
  • Allows synchronous VFS implementation (i.e. without Asyncify or SharedArrayBuffer).

or SharedArrayBuffer

Oh, that reminds me. I just recently wanted to port something NodeJS-ish that used sync node:crypto stuff to handle things to make it more cross env. friendly for Deno and web. so i wanted to switch it out to web crypto instead now that NodeJS has it too.

But it was to annoying to switch the entire codebase to be async-ified and breaking all the existing tests so i built await-sync to make the process a bit easier to use.

i think you @rhashimoto would maybe find this interesting and perhaps useful to transform async code to sync more easily.
...It uses SharedArrayBuffer behind the scenes. so it only works in backends and also web worker.

(a bit off topic, but i wanted to share it...)

Following this one

is it currently possible to save a sql.js db to opfs?