Description

khimaros

opened on Feb 6, 2021

Issue body actions

Wiki Page URL

https://github.com/ArchiveBox/ArchiveBox/wiki/Quickstart#2-get-your-list-of-urls-to-archive

Suggested Edit

document how to pull bookmarks and/or history from the Firefox Sync server.

this enables the user to simply bookmark pages they want to archive on their desktop and mobile browser and they will automatically be archived by archivebox.

here's what worked for me.

determine your Firefox Sync client id:

$ python3 -m pip install PyFxA

$ fxa-client --bearer -u <YOUR EMAIL> \
    --account-server https://api.accounts.firefox.com/v1 \
    --oauth-server https://oauth.accounts.firefox.com/v1
# ---- BEARER TOKEN INFO ----
# User: <YOUR EMAIL>
# Scopes: profile
# Account: https://api.accounts.firefox.com/v1
# Oauth: https://oauth.accounts.firefox.com/v1
# Client ID: <YOUR CLIENT ID>
# ---------------------------

import all of your bookmarks into archivebox:

$ git clone https://github.com/eNote-GmbH/syncclient

$ cd syncclient

$ python3 -m pip install -e .

$ python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> \
    get_records bookmarks \
        | jq '.[].payload | fromjson | select(.type == "bookmark") | .bmkUri)' \
        | archivebox add

this also works with a self hosted Firefox Sync server by setting the env var TOKENSERVER_URL="https://<YOUR SYNC SERVER>/token". you can configure this to run on a regular schedule eg. via cron.

with a bit of work, you could make this only archive bookmarks with a specific tag or folder.

Activity

1. pirate

   added

   size: easy

   good first ticket

   help wanted

   touches: docs

   on Mar 1, 2021
2. kominoshja commented on Nov 30, 2021

   

   kominoshja

   on Nov 30, 2021

   More actions

   Note: This requires disabling 2FA while setting it up
3. kominoshja commented on Nov 30, 2021

   

   kominoshja

   on Nov 30, 2021

   More actions

   When you issue python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_records bookmarks, you get something like

   "6J***********3Vk"
   "3_***********Iw7"
   "U1***********ah"
   

   therefore, jq fails. You'd need to run python3 syncclient/main.py -u <YOUR EMAIL> --client-id <YOUR CLIENT ID> get_record bookmarks --decrypt 6J***********3Vk | jq '.payload | fromjson | select(.type == "bookmark") | .bmkUri'

   I created the following script to iterate over the response and add bookmarks to archivebox (Keep in mind you need to change BATCH_SIZE to archive more than 4 at a time): https://gist.github.com/kominoshja/484dd83f978079daf6c3bacebb21fe37

   Reacted by khimaros
4. pirate commented on Jan 19, 2024

   

   pirate

   on Jan 19, 2024

   Member

   More actions

   Going to close this for now, I'll add a link to this issue from the README in case it helps ppl in the future trying to set this up.
5. pirate

   closed this as completedon Jan 19, 2024

Issue actions