REPL for your data

Use ethical solutions. Archive your data. REPL.

So we have heard this time and time again.

  1. Data is the new oil.
  2. Surveillance capitalism.
  3. The most profitable companies on the entire planet rely on your data1.

and so on. I believe the agenda for all these companies is to use data to power their AI research. And our privacy is just a casualty in this process. Despite all this news and "shocking" revelations that we hear almost every day in the news, many tend to stick around and use them, including me2. There are solutions however. And it gets better - what I described below require almost zero investment (except for your attention and time, of course).

Solutions that do not require deep technical knowledge

  1. Jump ship to the new wave of decentralized social networks, which are collectively called Fediverse3.
  2. Start using zero-knowledge and FOSS software services for example collaborative document editing4 and emails5.
  3. Prefer end-to-end encrypted chat applications which cannot be compromised6.

Solutions that need you to be a power user

If you are tech-savvy enough, you can start using OpenPGP with email using Enigmail. And for services you can also consider self-hosting it7 in low-power devices such as a Raspberry Pi. You can even get a URL to point to your server at absolutely no charge8.

REPL: Read, evaluate, print and loop

REPL is a jargon for consoles of all sorts: bash, python etc. In this case I apply this into our social media usage. Read what others post, evaluate your thoughts, share your thoughts into the platform, keep doing it. What is unique about REPL consoles is that data is ephemeral. What you type is not meant to last forever.

Therefore, I have started to cleanse my digital data footprint. I deleted my Facebook account several months ago. Right now I am deleting all my tweets10. My reasons are more than privacy oriented. I do not want blood on my hands on a platform that have a proven impact on how people think, how they vote and do things. And these platforms profit from selling analytics, and manipulate users using algorithmic timelines and recommendations. I am therefore progressively transitioning to more ethical services.

Unless you are in full control, you have to be always to be cautious while entrusting some of your data with a third-party service. Even if they are ethical now. I remember a time not so long ago, when I had a positive impression about Google and its "Don't be evil" tag line9. I keep archives of my Mastodon account11 and will soon add expiration to my posts. I also have this idea:

Appendix 1: Mass tweet deletion

So the commands described in the blog post10 seem to work great. Here is what I did following the instructions:

  1. Downloaded an archive from Twitter
  2. Copy and modify tweet.js into a proper JSON file.
  3. Extract the ID of all tweets.
  4. Sniff out the POST request as a cURL command using Firefox. I also added the silent flag -s for cURL. Save it as deletetweet.sh and make it executable.
  5. And a personal twist: send delete requests in parallel:
cat tweetstodelete.txt | parallel -j4 "echo 'Deleting {}' && ./deletetweet.sh {}" 2>&1 1>> delete.log &
tail -f delete.log

And the cURL command looks like:

curl -s "https://api.twitter.com/1.1/statuses/destroy/$1.json" ...

So now I am down from 1400 tweets to 544 tweets. And the difference is almost correct:

❯ wc -l tweetstodelete.txt
844 tweetstodelete.txt

In step 3, the following command was used to extract the tweets with no interaction

cat tweet.json | jq '.[] | select(.favorite_count == "0") | select(.retweet_count == "0") | select(has("in_reply_to_user_id_str") | not)  | .id' -r > tweetstodelete.txt`

To get a list of all the tweets:

cat tweet.json | jq '.[] | .id' -r > tweetstodelete.txt

Now we need something similar for undoing all the likes on Twitter.

EDIT: A short how-to "dislike" all your tweets.

Get a list of tweets to dislike from your archive in a similar approach. Take the like.js file from your archive and make it a proper JSON file.

cat like.json | jq '.[] | .like.tweetId' -r >! tweetstodislike.txt

The POST request is also similar and contains "destroy". However the tweet ID should be in the data / payload. Therefore the curl command looks like (note the $1 where the tweet ID gets substituted):

#!/bin/bash
curl -s 'https://api.twitter.com/1.1/favorites/destroy.json' ...lots of stuff here...\
  --data"id=$1&cards_platform=Web-13& ...even more stuff..."

Save the command in a file called disliketweet.sh and then,

cat tweetstodislike.txt | parallel -j4 "echo 'Disliking {}' && ./disliketweet.sh {}" 2>&1 1>> dislike.log &
tail -f dislike.log

Appendix 2: Unstar GitHub repositories

  1. Used bookmark-github-stars to export all my GitHub stars as html and imported into Zotero (you can also import into your browser). I did so because because Zotero also retains valuable metadata (programming language, license, description etc.).
  2. As Zotero does not automatically fetch metadata in its initial import, I wrote a few scripts which relies on pyzotero. The documentation is minimal, but you can reuse them if you want.
  3. Finally I run this script which relies on pygithub.

  1. Also known as GAFAM, the big four / five. Framasoft gave an excellent talk about it FOSDEM 2017 if you don't mind the French accent. 

  2. Including me. Even this very blog post is right now on GitHub pages, and this would seem like a contradiction. In my defence, I started blogging here before the acquisition. As much as I don't like MS products, GitHub doesn't look like the worst place in the world and blogs are not private places. 

  3. Such as Mastodon 

  4. Good alternatives are Cryptpad and Etherpad

  5. See alternatives of which I use ProtonMail. The only downsides that I think are: for extra data allowance and POP3/IMAP access you need a paid account 

  6. Google docs spreadsheet which compares digital communication protocols 

  7. Why you no host 

  8. Using Dynamic DNS technology. I use a subdomain name provided at afraid.org

  9. Which was removed in 2018 as they no longer follow it. 

  10. I followed @jlelse's tutorial to delete tweets. 

  11. Using mastodon-archive 

  12. This is an interesting essay on why you should expire your posts.