Link Search Menu Expand Document
NanBhas

Mendeley To Notion: Tractable Lit Review (II)

project workflow toolkit research mendeley lit-review notion-api notion open-source python

Ever wondered how to stay on top of scientific literature? Meet MendeleyToNotion, a Python-based, fully open-source package, that allows you to automatically keep track of your literature review collected on Mendeley in Notion via their APIs.


Table of contents
  1. Description
  2. GitHub Repository
  3. Usage
    1. Mendeley Setup
    2. Interfacing with Notion
    3. Running the script
  4. Requirements
  5. Directory Structure
  6. Sources

MendeleyToNotion in actionMendeleyToNotion in action

Description

This project allows you to export newly added or recently updated documents in Mendeley to your Notion database via the APIs provided by the two. If you’d like the export to happen as soon as you make a change in Mendeley, then you can run the script scripts/runMendToNotion.sh peridocially at a reasonable frequency via a crontab job.

GitHub Repository

You can find the all the assosciated code for MendeleyToNotion in this github repository

Usage

Update: This hasn’t been tested out after Mendeley’s Sept 1, 2022 changes. It is not clear how long the API will remain available for easy access to Mendeley data.

Mendeley Setup

  • Register an app on Mendeley’s developer portal (follow instructions online)
  • Obtain its clientID, clientSecret and redirectURL and add it to secrets/secrets_mendeley.json in the following format:
{
    "clientId": "your client ID", 
    "clientSecret": "your client secret", 
    "redirectURL": "your client redirect URL"
}

Interfacing with Notion

  • Register a private integration on your Notion workspace (follow instructions online)
  • Obtain its notionToken
  • Create a database on Notion to contain all the entries from Mendeley. Make sure it has the following properties. If you want to add more properties or remove, modify the function getPropertiesForMendeleyDoc and getNotionPageEntryFromPropObj in lib/port_utils.py.
Title property: Citation
Text properties: TItle, UID, Authors, Venue, Year, Abstract, Type, BibTex, Filename, ARXIV, DOI, ISSN, ISBN, PMID
Url properties: Filepath
Date properties: Created At, Last Modified At
  • Get its databaseID and add it to secrets/secrets_notion.json in the following format:
{
    "notionToken": "your notion token",
    "databaseID": "your notion database ID"
}

Running the script

  • Create a python conda env using requirements.txt
  • Run the python script src/mendeleyToNotion.py with --secretsFilePath argument as secrets/secrets_mendeley.py.
  • Authenticate your Mendeley app by logging in. It will automatically generate a token and add it to your secrets/secrets_mendeley.py.
  • Currently, the logic reads all items from the Mendeley API as an Mendeley Object iterator and reads all items in the Notion database. For each item in Mendeley, we check if it already exists in Notion. If yes, we check if the last modified time on Mendeley is AFTER the last edited time in Notion. If yes, we update the Notion entry. If not, there’s nothing to update. If the Mendeley item doesn’t exist in Notion, we create a brand new row in Notion.
  • You can periodically run this file again as a script scripts/runMendToNotion.sh using a crontab job to get periodic updates. For more information on Crontab, check out this reference.

Requirements

You can install all the requirements using the following command:

pip install -r requirements.txt
pip install -e .

Directory Structure

.
+-- docs/
|   +-- images/
|   |   +-- demo.png
|   |   +-- icon.png
+-- globalStore/
|   +-- constants.py
+-- lib/
|   +-- port_utils.py
|   +-- utils.py
+-- mendeley/
|   +-- auth.py
|   +-- session.py
|   +-- ...
+-- notebooks/
|    +-- trial.py
+-- scripts/
|   +-- runMendToNotion.sh
+-- secrets/
|   +-- secrets_mendeley.json
|   +-- secrets_notion.json
+-- src/
|   +-- mendeleyToNotion.py
+-- .gitignore
+-- environment.yml
+-- juyptext.toml
+-- README.md
+-- requirements.txt
+-- STDOUTlog_examples.txt

Sources

Note: I wanted to make use of the mendeley python sdk directly, but their auth code has issues with the refresh method. Instead, I have cloned the mendeley project and made changes to it locally, most notably in the MendeleyAuthorizationCodeTokenRefresher class in mendeley/auth.py

def refresh(self, session):
    #oauth = OAuth2Session(client=self.client, redirect_uri=self.redirect_uri, scope=['all'], token=session.token)
    #print(session.token)
    #oauth.compliance_hook['access_token_response'] = [handle_text_response]

    #session.token = oauth.refresh_token(self.token_url, auth=self.auth)
    session.refresh_token(self.token_url, auth=self.auth)
    #print(session.token)

If you use this in your work and want to adapt this code, please consider starring the github repo or forking from it!

You can cite it as:

@misc{ nanbhas2023mendeleyToNotion,
  title         = "Mendeley To Notion: Tractable Lit Review (II)",
  author        = "Bhaskhar, Nandita",
  howpublished  = "GitHub Repo, https://github.com/nanbhas/MendeleyToNotion",
  year          = "2023",
  url           = "https://web.stanford.edu/~nanbhas/projects/mendeley-to-notion/"
}