I've published the source codes for my website and the library notion-x that I built for it. However, I don't have time to provide detailed guidance on how to use them! In this note, I discuss the reasons behind this and some tips or problems I encountered during the theme development. I believe this is more beneficial than merely providing a step-by-step guide. Let's learn together and build a site like mine!
This post contains a large amount of text. I understand it may be difficult to follow, but I don't have another way to express these detailed ideas.
- I tried in order Jekyll β Gatsby β 11ty β Next.js. I end up with Next.js as a SSG + Notion as a CMS.
- If donβt want to use Notion, you can try Obsidian. However, this post isnβt for the link with this tool.
- Why Notion? β You cannot find another WYSIWSG that is more powerful than this one. It supports a quick full-text search and we can integrate it with Next.js.
- Why not Notion official APIs? β Itβs slow + limit the number of requests each second, you will meet too many 429 errors.
- To use unofficial APIs (the APIs Notion uses on their website), I customize react-notion-x to adapt the styles I want. I call this customized as notion-x. I use this libray for all my projects.
- Why Next.js? β Itβs react based, itβs quick and we can use Vercel for deploying our app easily and quickly.
- For different types of posts/collections, I use different Notion Databases. For example, notes, topics, projects, tools, bookmarks have different databases.
- The biggest problem you may encounter when deploying your site is the Serverless Function Execution Timeout. Vercel and other free services only support 10s of SSR. You have to make your codes run faster as much as you can. Thatβs why we use unofficial APIs and other tips.
I wrote this section based on my memory of the frameworks I've tried. They may have been updated since the time I wrote this section. It would be best to check their official sites for any new changes.
I initially created my site with Jekyll, popularly known as a static site generator, when you search for "creating your own personal website". This is why from version 1 to version 3, I used Jekyll to develop this site.
However, as the size of the notes grew and the need for regular content changes increased, the build time also escalated. Thankfully, Jekyll enhanced their incremental regeneration, which was a game-changer. It allowed for building only the recent changes, significantly reducing the time and improving the workflow!
I soon realized that extensive customization of my site required understanding of Ruby, which I found challenging. Consequently, I migrated my site from Jekyll to Eleventy, a framework that offers in-depth customization using only JavaScript. I've documented my process of building an 11ty website from scratch in this detailed note.
Another appealing feature of Eleventy is the ability to use Nunjucks to write your site's template. This language is similar to what Jekyll uses, so the transition wasn't overly time-consuming. In fact, it took me just a month and a half to fully migrate from version 3 to version 5. As for why I didn't migrate to version 4, please continue reading.
Before discovering 11ty, I experimented with Gatsby, a rapidly growing static site generator (SSG) framework. I was highly impressed by its rendering speed. Even minor changes in the code could be seen instantly in the browser without any latency, a feature common to single-page applications and other React apps.
I had completed about 60% of a site build with Gatsby when I encountered difficulties with inserting images in markdown files at custom positions and sizes. I needed my notes to be flexible to effectively express my ideas, while maintaining a clean and clear design. Due to these issues, I abandoned Gatsby and switched to using 11ty instead. I also noted the whole process in this note. Moreover, if you want to migrate from Wordpress to Gatsby, another note of mine may be useful for you too.
Having used 11ty for nearly two years and recently discovering Notion, I've been taking most of my notes using Notion. However, a significant challenge arose because 11ty primarily uses markdown files for content creation (other sources can be used, but it's not straightforward). Whenever I need to publish notes, I must convert them from Notion to markdown and maintain synchronization between two locations, Notion and markdown files. This process was inconvenient and time-consuming.
I wished for a mechanism where I could make notes on Notion, and any changes I made would automatically update on the site. That's when Next.js and the idea of using Notion as a CMS occurred to me. The version 6 was born.
Here's what I appreciate about Notion:
- It's free for personal use, with unlimited notes and blocks.
- It offers a variety of block types for note-taking in many colors. My favorites are "Toggle," βCalloutβ, "Column", βCodeβ and "List."
- It allows for quick pasting and uploading of images.
- It provides APIs for creating unique features.
- It enables collaboration on notes with others.
- AI features can help you improve your notes without breaking the format.
- Itβs cross platforms (mobiles and desktops).
- Most importantly, it provides full text search for all your notes.
So, what more could you want from a note-taking platform? While others may prefer Obsidian, Notion meets all my current needs.
Indeed, I wish they would add two more features: a table of contents in a sidebar and offline support for their current desktop app.
Notion offers excellent official APIs for its blocks. These APIs enable you to retrieve and modify your notes with ease. In fact, I created my own library to convert all their block styles into components in Next.js using these official APIs. You can check it out here.
However, one significant issue with using Notion's official APIs is the "Too many requests" (error 429) problem due to request limits. This error often occurs because you need to fetch all the blocks and notes before converting them into HTML files using Next.js. Moreover, due to Next.js's mechanism, you may repetitively call the same request for different types of posts/items.
Another weakness of the official APIs is that they do not fully support the styles used on the Notion site. For example, in Notion, you can adjust the size of each column in the column block. However, in the official API, all columns have the same size.
So, if we can't use the official API, what should we use? The answer is an unofficial one. I recommend react-notion-x. The author has designed it as a comprehensive solution for requesting all types of Notion blocks. This library attempts to replicate the styles present on Notion. While this is an advantage, it can also be a disadvantage if you want to customize your size like I did.
A major drawback of unofficial APIs, in comparison to official ones, is the inability to filter requests like the official ones can. For instance, if you want to fetch posts based on categories and also limit the number of posts per request, you can only do that with the official APIs. So, using both official and unofficial is the best strategy!
- It is extremely fast, with retrieval speed equal to that of the Notion site.
- It fully supports the styles of the block. What you see on the Notion site, you can achieve the same in the response of the unofficial APIs.
- You can utilize the unofficial APIs to create a free full-text search feature. This code gives you an idea of how to write an API in Next.js for getting search results from Notion using unofficial APIs. This code and this one give you an idea of how to using the API you create in Next.js to display the search result on your page.
I chose to create my own notion-x, a customized version of react-notion-x, to fully personalize my site with my preferred styles. This library also utilizes Tailwind CSS for styling. If interested, you can view a demo of the components in use within this library.
I created this library to use its styles across various websites (dinhanhthi.com, math2it.com, dinhanhhuy.com, β¦), instead of modifying styles within individual projects.
I chose Next.js primarily because I'm fond of React, although I also work with Angular. Among the frameworks available, I find Next.js and Gatsby to be the best. I tried Gatsby but encountered some limitations, which naturally led me to choose Next.js.
Furthermore, Next.js provides a comprehensive environment for both backend and frontend development. It enables us to create full-stack applications with ease, akin to using Tailwind for styling.
An important point to note is that you have to decide which version of Next.js to use. Starting from version 13.4, a new paradigm called App Router is used, unlike previous versions which used Pages Router. The App Router inherently has more advantages and incorporates more contemporary techniques than the Pages Router.
I initially started building this site using Pages Router, but later switched to App Router and quickly grew fond of it.
Another reason I appreciate Next.js is due to the free-tier hosting for apps offered by the company behind it, Vercel. The free usage they provide is more than sufficient for running this site and my project Math2IT.
If you're just looking to learn CSS by building something, I wouldn't recommend using Tailwind CSS. However, if you want to boost your development process, I highly recommend it.
I love Tailwind because it helps keep my code clean and consistent. I don't have to worry about adjusting the spacing between two divs, as they will be the same regardless of their location. It's all about maintaining consistency!
One downside of using plain CSS is having to name classes and decide which ones should be nested. This can make your code look messy and make it harder to track down bugs. Of course, there may be some who disagree with me on this point.
Tailwind also provides a playground where you can experiment with styles before applying them to your project. It's a great feature!
One thing I don't like about Tailwind is that you can't try out classes directly in the browser if they're not used in your code yet.
Actually, the APIs we're going to use are also from Notion, but they are not publicly available. When you view your note on the browser, it's because you're already signed in and using your credentials to request the content from Notion using their hidden APIs. To see the requests you can make with your Notion notes, simply open the Development Panel of your browser and check the Network tab.
For authorization, you will need two things -
token_v2 and notion_user_id. You can retrieve them in the Application tab β Storage β Cookies β https://www.notion.so.In the header of your request, you need to set,
1{
2 "Content-Type": "application/json",
3 "cookie": "token_v2={{your_token_v2}}"
4 "x-notion-active-user-header": "{{your_notion_user_id}}"
5}To fill in the body of the request, simply replicate the actions performed in the "Payload" tab of each request you see in the browser. That's all!
You can use Postman to quickly test the APIs and check their responses.
I create different databases for different types of data on this website. When you create a new database, check the request
queryCollection in the Network tab. We need to find the corresponding keys to the properties in the database.One specific case, if you see the key like
a\\b;, it's actually a\b;!Update: The key mentioned above actually refers to the database property ID. You can obtain these property IDs using Retrieve a database. Note that the API returns encoded IDsβfor instance, you'll get
_%60Gn instead of _`Gn. The latter (_`Gn) is what we need!Write down these keys and keep them in your
.env.local file. You can manipulate the response object to obtain the desired value. Check this method as an example.
One important thing to note is that Notion automatically refreshes the link of any media uploaded to their server every hour. If you have a private database, the images in your posts will display as "not found" after one hour from the time you built the site. The new URL cannot be updated automatically unless you make your database public.
When creating a library similar to notion-x, there are different options available:
- Create a library and integrate it into your project as an npm package. You can install it using
npm i your_packageoryarn add your_package.
- Add your package to your project as a submodule. Whenever you make changes, you first push the changes to the package, and then update the submodule in your project.
- Clone your package directly into your project and exclude the cloned folder from the main project in Git.
Based on my personal experience, I prefer the second method because it is faster than the other two methods when making changes to the package. In the first method, you have to remove the package and reinstall it, which can impact other packages in your project and takes more time. With the third method, you may encounter conflicts between the git of your project and the git of the package placed inside your project.
As mentioned earlier, one significant reason I chose Notion as a CMS over other SSG engines is its default full-text search.
On the Notion site, you can promptly search for anything in your Notion workspace by pressing
ctrl + K (or cmd + K on macOS). It's fast, and it displays almost everything.This is why unofficial APIs are favored in this case to fetch the contents. You make the same requests as on the Notion page when you search for something. You can try this feature on my site. I've implemented it the same way as on the Notion site, with more advanced functionality in case you want to use your keyboard to navigate between results.
Let's discuss how to implement this feature. You need to:
- Create an API on your Next.js site. This API will call the unofficial APIs and return the search results. You'll have to modify the results to fit your types. This is necessary to conceal your Notion credentials. See this file for an example.
- On the client side, use swr to call the API you just created based on the query users type in the search bar. See this file as an example.
As you can see, there is a Bookmarks page on this site. I use this page to store and introduce all of my bookmarks to the visitors.
By default, it lists all the bookmarks I've saved (also on Notion) with pagination. If you search for something, it displays the found bookmarks instead.
One special thing is that I donβt have to input the title, description, and images of each bookmark; they will be automatically fetched and filled in the Notion database.
Letβs talk about the ideas behind this.
- We create a separate Notion database for storing these bookmarks. The most important property is βurlβ. All other properties (title, coverUrl, description) are automatically fetched and filled. Check this database as an example.
- In your Next.js site, when rendering the bookmark page, you get all the URLs from the database and fetch their metadata using the package url-metadata (or similar packages). After having the metadata, we update this information on the Notion page using the Notion APIs. Check this function as an example of how to do this step.
One important thing in this step is that we only fetch the metadata if we didnβt input these on the database. This is for the bookmarks we would like to write our own descriptions or for ones not having any metadata.
Another special thing is that we donβt perform this step for every request, we only get the metadata and update them on Notion once. It significantly reduces the number of requests to Notion. Itβs quick!
- For the search feature, just do the same things as the full-text search feature of the page (read section βFull text searchβ for more). Check this file and this file as an example.
During the initial development of this site, I began with a small number of posts, and each post had a modest amount of content. However, this approach became problematic when I transitioned to a real database containing many posts, each with numerous blocks. Too many 429 errors (too many requests) started to occur.
Building and re-rendering times also increase when working with a real database. This can lead to complications when deploying your site on a real server (as opposed to a more powerful local machine).
Therefore, it's crucial to consider the actual size of your database during development. I won't provide specific solutions here, as they are mentioned elsewhere in this post.
As you can see, I use many icons on this site, primarily from the react-icons package. However, I didnβt install this package directly. This is because when you use a third-party package like react-icons or lodash, you may inadvertently import their entire package despite only using some of their elements. This can substantially increase your project's weight and impact building and re-rendering times.
To mitigate this issue, you can utilize specific techniques provided by each package or use the Modularize Imports feature of Next.js.
In my own experiments, I used Modularize Imports for lodash and created separate icon elements in my notion-x. These icons were copied from the react-icons package.
When working with a Notion database, loading time and operational speed can decrease as the database size increases. Therefore, it's beneficial to create separate databases for different purposes.
For instance, I created separate databases for notes, projects, bookmarks, and tools on my site. This approach simplifies requests and reduces the time spent working with Notion.
As mentioned in other sections, to utilize full-text search or display images uploaded to Notion on your site, you need to make your database public.
If you prefer not to make your database public, consider these alternatives:
- Don't upload images directly to Notion! Instead, use a third-party service like imgurl or Cloudinary. Imgurl is a completely free service, but it doesn't support GIF images. Cloudinary offers sufficient usage with their free tier, and you can customize your images by modifying the URLs.
- If you insist on uploading images to Notion, consider using the same approach as in the "Creating the Bookmarks Page" section. Automatically detect and upload the same image to third-party image services using the APIs provided by these services. Both imgurl and Cloudinary offer APIs for this.
With this approach, every time you upload an image to Notion, Next.js detects it and downloads the image. It then uploads the image to imgurl/Cloudinary and updates the URL in your Notion block. As a result, images in your Notion post will have the URL of imgurl/Cloudinary.
I'm looking for a free option to deploy my site, and Vercel is all I need. Using the official APIs like the time it takes to build and re-render content if there are any changes is too long (more than 10 seconds for free). That's why I need something faster for the re-rendering process, like the unofficial APIs.
To overcome the "Serverless Function Execution Timeout" problem, you need to optimize your code to run as fast as possible. The use of unofficial APIs can be a game changer. Another important point is to avoid using something like plaiceholder to generate photo previews on your site, as this process takes a long time due to the need to download the image before generating the preview.
Before using notion-x (unofficial APIs), I was using notion-nextjs-lib (official APIs) and encountered the issue of frequent timeouts on Vercel (you can check the error in the Logs tab of the Vercel Dashboard of each project).
Another idea for deploying your site without worrying about the build time is to use the Static Exports feature of Next.js. This method is similar to other Static Site Generators in that it exports the entire site into HTML files, which can then be hosted. However, it has a major drawback: you cannot automatically update the changes made in Notion.
Whenever you make a change in Notion, you need to rebuild the entire site and redeploy it. One technique is to identify the files that have been changed and only copy those files to the folder containing the HTML files. I have used this technique for my Math2IT project before I started using the unofficial APIs.
Vercel (and similar services like Netlify, AWS Amplify, etc.) allow you to build your static site on their servers. To deploy changes to your site, simply trigger the build process on their service. Note that the build time may be longer than on your local machine, depending on the power of the server you choose (free tiers often have weaker servers).
While Notion offers a compelling platform for managing content, there are two limitations that arise when using it as a CMS:
- Due to the rapid pace at which content is fetched using code, you may encounter frequent 429 errors (too many requests). This occurs because the Notion server detects an unusually high volume of requests and imposes a per-second limit. As a result, your site may display 404 errors (not found) instead of the intended content. To address this issue, inform visitors to refresh the page, which should resolve the error and load the correct content.
- Another common issue is the occurrence of 404 (not found) errors for posts. This is likely due to the server-side rendering of posts, which involves the server generating the content upon each user's request. In some instances, the content may not be immediately available, leading to a 404 error. To mitigate this, instruct users on the 404 page to refresh the page, which should reload the content correctly.
Upgrading to Vercel's pro version may alleviate these limitations. While I haven't personally tried it, it appears to be a viable solution.
A notable limitation is the requirement to make your Notion database public. This implies that anyone with the database URL can access and view your content in Notion's default style. However, this issue is somewhat mitigated by the fact that Notion automatically hides your site from search engines. To enable search engine indexing, you would need to upgrade to their Pro plan.
I think that's enough reading for today. There are still many other aspects to consider when developing this site, but I can't remember them all at the moment. I will continue updating the content of this post, and I hope you find it useful. Thank you.
I wrote this section based on my memory of the frameworks I've tried. They may have been updated since the time I wrote this section. It would be best to check their official sites for any new changes.