-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Test * Update gatsby-config.js * Changes search result text * Fix gatsby-config.js and title * Change meta description * Fix sitemap.xml for broken links * Update landing page: Key Update and image * Create Blog “2024-04-21-how-to-integrate-langchain-with-apache-age-for-graph-database-applications” * Update Blog “2024-04-21-how-to-integrate-langchain-with-apache-age-for-graph-database-applications” * Update Blog “2024-04-21-how-to-integrate-langchain-with-apache-age-for-graph-database-applications” * Update blog, cms * Update blog * Fix spelling error * Activate "blog" * Update download page, modified blog * modified footer, download * Update landing image and blog scss * modified: src/components/styles/Blog-post.scss * modified: src/pages/download/index.md * modified: src/templates/blog-post.js * Create Blog “2024-04-23-beyond-chatgpt-how-to-maximize-the-use-of-chatgpt-with-interactive-graph-models” * Create Blog “2024-04-23-combating-cyber-attack-with-apache-ages-link-analysis” * Create Blog “2024-04-23-from-data-to-connections-leveraging-hyperconnectivity-in-e-commerce-data-part-1” * Create Blog “2024-04-23-learn-machine-learning-with-graph-in-hyperconnected-data-part-2” * Create Blog “2024-04-23-graph-database-in-postgresql-apache-age” * Create Blog “2024-04-23-comparison-of-apache-age-postgraphile-and-hasura” * modified: static/admin/config.yml * modified: src/components/styles/Blog-post.scss modified: src/components/styles/BlogRollTemplate.module.scss modified: src/templates/blog-post.js * Update Blog “blog_240401” * Match file name format * deleted: src/pages/blog/blog_240401.md * Change button color * modified: static/sitemap.xml * modified blog * modified: src/pages/404.js modified: src/pages/index.md * Create Blog “2024-04-25-discovering-the-inner-workings-of-postgresqls-data-allocation” * Create Blog “2024-04-25-seamless-data-migration-migrating-apache-age-data-between-different-versions-of-postgresql” * Create Blog “2024-04-25-how-to-deploy-apache-age-docker-image-on-aws” * modified: static/admin/config.yml * modified: src/pages/blog/ modified: src/pages/blog/2024-04-25-how-to-deploy-apache-age-docker-image-on-aws.md modified: src/pages/blog/2024-04-25-seamless-data-migration-migrating-apache-age-data-between-different-versions-of-postgresql.md
- Loading branch information
Showing
6 changed files
with
320 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
63 changes: 63 additions & 0 deletions
63
...log/2024-04-25-discovering-the-inner-workings-of-postgresqls-data-allocation.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
--- | ||
templateKey: blog-post | ||
title: Discovering the Inner Workings of PostgreSQL's Data Allocation | ||
date: 2024-04-25T08:29:55.344Z | ||
description: PostgreSQL is a true gem among open-source relational database | ||
management systems. Its impressive robustness, flexibility, and scalability | ||
make it a favourite choice for many developers and businesses alike. One of | ||
the most crucial components of any database system is how it manages and | ||
allocates data. In this article, we'll take a deep dive into PostgreSQL's data | ||
allocation process to reveal its inner workings and discover how it ensures | ||
optimal performance and reliability. | ||
featuredpost: true | ||
featuredimage: /img/logo.png | ||
--- | ||
<!--StartFragment--> | ||
|
||
PostgreSQL is a true gem among open-source relational database management systems. Its impressive robustness, flexibility, and scalability make it a favourite choice for many developers and businesses alike. One of the most crucial components of any database system is how it manages and allocates data. In this article, we'll take a deep dive into PostgreSQL's data allocation process to reveal its inner workings and discover how it ensures optimal performance and reliability. | ||
|
||
1. The Art of Data Storage: | ||
|
||
PostgreSQL stores data in tables, with each table containing data rows. Each row is made up of one or more columns representing various attributes of the data. When you create a PostgreSQL table, you must define its schema, specifying data types and constraints for each column. It's like giving your data a personality and a set of rules to follow. | ||
|
||
2. The Science of Storage Management: | ||
|
||
PostgreSQL's storage management strategy involves breaking data into smaller chunks, or pages. Each page is like a container with a fixed capacity, and PostgreSQL organizes data by placing it into these containers. Just like arranging items in multiple containers makes it easier to manage them, PostgreSQL's page-based storage allows for efficient data organization. | ||
|
||
3. Tablespaces: A World of Possibilities | ||
|
||
PostgreSQL organizes data into tablespaces, each representing a physical location on the disk where data is stored. A PostgreSQL database can have multiple tablespaces, allowing you to distribute data across different physical disks or file systems. This feature enhances performance and enables better control over storage resources. An example is having a house with multiple rooms, each with its unique purpose. | ||
|
||
4. Heap Files: The Foundation of PostgreSQL's Data Storage | ||
|
||
PostgreSQL stores data in heap files when you create a table. A heap file is an uncomplicated append-only file containing a series of data pages. When you insert new rows into a table, PostgreSQL appends the data to the heap file's pages, ensuring that new data is always written sequentially. This approach reduces fragmentation and optimizes disk I/O performance for write operations. It's reminiscent of having a notebook where you write your thoughts in chronological order. | ||
|
||
5. Page Structure: The Building Blocks of PostgreSQL's Data Allocation | ||
|
||
PostgreSQL employs a page structure within each heap file to organize data efficiently. Each page consists of a header, containing metadata about the page, followed by the actual data rows. The header includes information such as the page number, the number of items stored on the page, and a pointer to the next page. Imagine having a folder with labels for each paper inside. | ||
|
||
6. The Magic of Data Allocation: | ||
|
||
PostgreSQL automatically manages data allocation when you insert data into a table. When a new row is added, PostgreSQL determines the appropriate page to allocate the data. If the existing page has enough space to accommodate the new row, it will place it there. Otherwise, it will create a new page and link it to the existing pages to form a chain. It's like having a magician that knows all the tricks to make your data disappear and reappear in the right place. | ||
|
||
7. Free Space Map (FSM): The Map to Your Data's Destination | ||
|
||
To keep track of available space within each page, PostgreSQL maintains a Free Space Map (FSM). The FSM records which bytes within each page are used and which ones are still available for data allocation. When inserting or updating data, PostgreSQL checks the FSM to find the page with sufficient free space. It's like having a treasure map that leads you to the right spot. | ||
|
||
8. Indexes and Data Allocation: The Key to Your Data's Acceleration | ||
|
||
PostgreSQL's indexing mechanism plays a vital role in data allocation efficiency. When you create an index on a table, PostgreSQL creates a separate B-tree data structure that allows for faster data retrieval based on the indexed columns. The presence of indexes influences how PostgreSQL allocates data to maintain optimal access and search performance. It’s the same as having a GPS that guides you to the quickest route. | ||
|
||
9. Apache AGE: Expanding PostgreSQL with Graph Database Functionality | ||
|
||
While PostgreSQL excels in managing relational data, its capabilities extend into the realm of graph databases, thanks to its integration with Apache AGE (A Graph Extension). Apache AGE is an extension that transforms PostgreSQL into a graph database, allowing it to handle complex, connected data structures alongside traditional relational data. This integration offers the best of both worlds – the robust, tried-and-tested features of PostgreSQL for relational data, and the flexibility of graph database capabilities for handling interconnected data. Think of it as adding an advanced toolset to an already well-equipped toolbox, making PostgreSQL not just a database, but a versatile data processing engine. | ||
|
||
With Apache AGE, PostgreSQL becomes an even more powerful option for developers needing to manage diverse data models, from social networks to recommendation systems, without sacrificing the reliability and performance PostgreSQL is known for. This seamless blending of technologies demonstrates PostgreSQL's adaptability and its commitment to evolving with the changing landscape of data management. | ||
|
||
\*\** | ||
|
||
In conclusion, PostgreSQL's data allocation process is a fascinating and efficient system that ensures optimal performance and data organization. By dividing data into manageable pages, employing tablespaces for better storage management, and utilizing indexing mechanisms, PostgreSQL offers a reliable and scalable solution for handling large volumes of data. Understanding how PostgreSQL manages data allocation is crucial for database administrators and developers to optimize their applications and make the most out of this powerful database management system. Let's build a better world of data together! | ||
|
||
|
||
|
||
<!--EndFragment--> |
89 changes: 89 additions & 0 deletions
89
src/pages/blog/2024-04-25-how-to-deploy-apache-age-docker-image-on-aws.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
--- | ||
templateKey: blog-post | ||
title: How to Deploy Apache AGE Docker Image on AWS | ||
date: 2024-04-25T08:37:04.997Z | ||
description: > | ||
Apache AGE® is a PostgreSQL extension that provides graph database | ||
functionality. Apache AGE provides graph data processing and analytics | ||
capability to all relational databases. | ||
This article will explore how to set up Amazon Web Services (AWS), deploy the Apache AGE Docker image, and query the PostgreSQL database running in the Docker container on Amazon Web Services (AWS). | ||
featuredpost: true | ||
featuredimage: | ||
--- | ||
<!--StartFragment--> | ||
|
||
Apache AGE® is a PostgreSQL extension that provides graph database functionality. Apache AGE provides graph data processing and analytics capability to all relational databases. | ||
|
||
This article will explore how to set up Amazon Web Services (AWS), deploy the Apache AGE Docker image, and query the PostgreSQL database running in the Docker container on Amazon Web Services (AWS). | ||
|
||
Prerequisites | ||
|
||
1. AWS Account: If you don’t already have one, sign up for an AWS account at [Amazon Web Services](https://aws.amazon.com/). | ||
2. Docker: Install Docker on your local machine. You can download & install Docker from [here](https://www.docker.com/get-started). | ||
|
||
|
||
|
||
Step 1: Set Up AWS Account | ||
|
||
1. Log in to AWS: Access the AWS Management Console using your AWS account credentials. | ||
2. Create an IAM User: | ||
3. 1. Navigate to the IAM (Identity and Access Management) dashboard. | ||
2. Click on “Users” & then “Add user.” | ||
3. Provide a username and select “Programmatic access” to generate an access key and secret key for Docker authentication. | ||
|
||
Step 2: Deploy Apache AGE Docker Image | ||
|
||
1. Open a terminal on your local machine. | ||
2. Pull the Apache AGE Docker image from Docker Hub: ![](https://lh7-us.googleusercontent.com/6nFRHrsfi6mHtkh15ID0WR3ac8tdiZ7WLCbJYQQojyYbswAm5Q8bbFLhLNL85ZnYGaqEbEcI_fkt64yygvZ-2DzfG9pui4thQM9gPF24r7ZC_WzZ0vzTmdO-t5o4wc294Xey3A7IV98HAnr7Bg1Jxw) | ||
3. Run the Docker container locally to verify its functionality: ![](https://lh7-us.googleusercontent.com/0TgaSXMeJe3Ypped8Q0ibqZe_x-9X8Df7HVPcr7ABIhU8XiS8ul9tFQxifHeb3bHm9eEW3LzoirQMwNDW-Nuz2X4kADDyIn2j9lLBeX1drMIZuBjqq6BrKSpg4sv5-kqIc-5tjC_mY51605MhzQ64Q) | ||
4. Tag the Docker image with the Amazon Elastic Container Registry (ECR) repository URL: ![](https://lh7-us.googleusercontent.com/axtCSLN2EpvE3BKjHI18uPM0uIPW7BOQy9MpP5d91XPOjbiIkOVq_GtGq_QXnmWfuzD4odGwvjb4WMacZFET_cGmJV3fPfgrxHR643XT5Zr9tC3V51XE5i42MdwJMLw95lPpgcmPQZUgp84YR-6_sA) | ||
5. Authenticate Docker to the Amazon ECR:![](https://lh7-us.googleusercontent.com/4ofNVcRhkH1dZlXIA6gHs5wvsMx5wwtgjUT1TgrLtzjvPaFNPhBCoCv0NkPXFxOeU-0ECSEKChiDw5lLz-Uw3QfhB3ENN7tzfNqyCp7n4xVF6XRxfelSF_pTf9M813LUpi94xlJGXjfSSH1nMpjrvw) | ||
6. Push the Docker image to your Amazon ECR repository: ![](https://lh7-us.googleusercontent.com/AvJD8NMaZH2r8RlWhkNBZps-0FVryeEuvdm7Orr8zcwwDCoOWDhVMJfp-tUlbkZh4XJAvm4lMHegwv1fWVnxxR0aMjOSvXvLgwysH3XOVCZ0TVP9TVTu6eGtKxkxt6YfN8QhG_n8nCMnV9ZamjKESg) | ||
|
||
|
||
|
||
Step 3: Launch the Amazon EC2 Instance | ||
|
||
1. In the AWS Management Console, go to the EC2 Dashboard. | ||
2. Click “Launch Instance” and choose an Amazon Machine Image (AMI) of your choice. | ||
3. Configure the instance settings: | ||
4. * Choose an instance type. | ||
* Configure instance details and storage. | ||
* Add tags if necessary. | ||
* Configure security group rules to allow incoming traffic on port 5432 (PostgreSQL). | ||
5. Review the configurations and launch the instance. | ||
|
||
|
||
|
||
Step 4: Deploy Docker Image on Amazon EC2 | ||
|
||
1. SSH into your Amazon EC2 instance using the key pair associated with the instance:![](https://lh7-us.googleusercontent.com/uMdyOrdtw_eNyJe2B2-u56su4FxhHSfic4RfrUQnKhGvEj_xbPzwqO5nm8Pj8BppZhaJa--r1-HMuKoVUZSfNrKM_qf2WzGj_Fg3KWgWsVZnqRRrj1BMl0g7IoKwp_I_XXpyIBP0F5OyRA30XFfPkg) | ||
2. Install Docker on the EC2 instance: | ||
|
||
![](https://lh7-us.googleusercontent.com/NYK-wsc0WeuaTnwQ7t_4oWkEIYl-iEyUOBFN1FilZHSOcW8UNGNuoWBH3qZgvaFoGlPANR3WVXRg_O14Ot7DAPOweFFP8exvtMF1ycB-oJNn271T7157Ucqg8QbA-7_HfUsxzpn3XjjUGxoSaoGlLg) | ||
|
||
![](https://lh7-us.googleusercontent.com/7CctADC_EVZSDC1XThYj_qo631j1Fp_LKuX5bSvFzu52i0fDiMl0ol7P1tybkrES6hHC9HPsffQaMI_5m7u218M1wVaewBFReYmzfiRaH-ftWM0vCRQ0Qv88LsOQmScwHvBumhvNvhKP8oXNZHWvDg) | ||
|
||
![](https://lh7-us.googleusercontent.com/tsL9-TfHumrTK-lYj8xLz7F0Jrgio_Z7AYzl6HG4jK_U2qUyAA7dvxJ1dW7lLdK9az1jw0YdiT2wRIuKJbd_nsWxDRntAvMnPQ8vfVpBcWh8be6mxmRdhJAQ9FAWSUYuCZvFZkvKC8KfWpS09J0ycA) | ||
|
||
![](https://lh7-us.googleusercontent.com/orfuWcUbbs_AnDRRrXoP_SphDdPRZ8oUvxrYKSGh1oIqLiRYy4wmr1IC8LnOQEm51geFSVvTEu8OaM8EzIaB_brmGui3u-O5LYeC6YK6WQ8E8pKghzsJx40Zrjhi0fM_E9DGTV8DoYxvlqxhpFWFZw) | ||
|
||
3. Pull and run the Apache AGE Docker image on the instance: | ||
|
||
![](https://lh7-us.googleusercontent.com/p8OdSdB632x4DGcTekVMo2b1RlEFCX9VKdDhxMfUOaSjfwBpGRRjhZVlz2DH3866Xx22LT9jb_w5A75eUe9JcMnrinVpVequRrjRZWsrITS24i_SDpMUm5VMpkb3BOPRBUFJrdkBAyUwuKJ7z0ahnA) | ||
|
||
Step 5: Access & Query PostgreSQL Database | ||
|
||
1. Install the PostgreSQL client on the EC2 instance:![](https://lh7-us.googleusercontent.com/yPKtED71f06maB_4FPymzRcljCkgjC3-yyCKnckA1vEulBIU5mprhKyQUAPwVPQ-__ZFWcN6lWxtTTdPDO5HBl_LTpcYCB6KZG32_bTcV3EVTJGzsMe_pyb8EVGLbbhNjSCw01_az0z8Qt6W6RklOg) | ||
2. Connect to the Postgres database running in the Docker container:![](https://lh7-us.googleusercontent.com/yv_uYVG0g_Oe8WE3Oc7xpZSbd3UhhOk5WRVRxMG5a7uwbyHOBS2ZJ5cB45z7slHIJfO6rseFccjInwTYH9jZLWPeiVqs4Qm4OLJQ5TMjj4Y_qAlVrEWvy1WkUqRE_yt83NyZRBVBsPFkRosMcKLz7g) | ||
|
||
You are now connected to the PostgreSQL database. Execute SQL queries and interact with the database as needed. | ||
|
||
Conclusion | ||
|
||
By following this comprehensive guide, you have successfully set up an AWS account, deployed the Apache AGE Docker image on an Amazon EC2 instance, and accessed the PostgreSQL database within the Docker container. This setup provides a secure and scalable environment for your data encryption and storage needs. | ||
|
||
|
||
|
||
<!--EndFragment--> |
164 changes: 164 additions & 0 deletions
164
...migration-migrating-apache-age-data-between-different-versions-of-postgresql.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,164 @@ | ||
--- | ||
templateKey: blog-post | ||
title: "Seamless Data Migration: Migrating Apache AGE Data Between Different | ||
Versions of PostgreSQL" | ||
date: 2024-04-25T08:33:00.464Z | ||
description: > | ||
Apache AGE serves as an open-source graph database, akin to Neo4j. Although | ||
they share similar functionalities, they also exhibit significant differences. | ||
AGE stands out as a PostgreSQL extension which leverages the capabilities of a | ||
relational database (RDBMS), enabling users to seamlessly integrate both | ||
relational data (tables) and graph data, which is a key strength setting it | ||
apart from Neo4j. | ||
featuredpost: true | ||
featuredimage: | ||
--- | ||
<!--StartFragment--> | ||
|
||
Apache AGE serves as an open-source graph database, akin to Neo4j. Although they share similar functionalities, they also exhibit significant differences. AGE stands out as a PostgreSQL extension which leverages the capabilities of a relational database (RDBMS), enabling users to seamlessly integrate both relational data (tables) and graph data, which is a key strength setting it apart from Neo4j. | ||
|
||
|
||
|
||
AGE is available in various versions, each compatible with specific PostgreSQL versions. One common challenge users encounter is data migration when transitioning from one AGE version to another. Fortunately, PostgreSQL provides a utility called "pg_dump", which seamlessly functions with AGE for data migration. | ||
|
||
|
||
|
||
Let's exemplify the data migration process by creating a graph with some nodes: | ||
|
||
|
||
|
||
SELECT create_graph('test'); | ||
|
||
|
||
|
||
NOTICE: graph "test" has been created | ||
|
||
create_graph | ||
|
||
\-------------- | ||
|
||
|
||
|
||
(1 row) | ||
|
||
|
||
|
||
SELECT * FROM cypher('test', $$ | ||
|
||
CREATE (:Person {name: "John Doe"}) | ||
|
||
CREATE (:Person {name: "Jane Smith"}) | ||
|
||
$$) AS (res agtype); | ||
res | ||
\----- | ||
(0 rows) | ||
SELECT * FROM cypher('test', $$ | ||
|
||
MATH (n) | ||
|
||
RETURN n | ||
|
||
$$) AS (res agtype); | ||
res | ||
\---------------------------------------------------------------------------- | ||
{"id": 1, "label": "Person", "properties": {"name": "John Doe"}}::vertex | ||
{"id": 2, "label": "Person", "properties": {"name": "Jane Smith"}}::vertex | ||
(2 rows) | ||
Now, we have a graph called "test" containing two vertices (keeping it simple). | ||
Using pg_dump for Backup: | ||
To create a backup, open a new terminal in your current postgreSQL directory and execute the following command: | ||
bin/pg_dump -d postgres > age_backup.sql | ||
Here, "postgres" represents the name of the database (which might differ in your case), and "age_backup" is the chosen name for this demonstration. The generated "age_backup.sql" file will contain all the necessary queries to restore this backup whenever needed. | ||
Now, if you open “age_backup.sql”, which will be in your PostgreSQL directory, its initial content will resemble this: | ||
\-- PostgreSQL database dump | ||
\-- Dumped from database version 13.9 | ||
\-- Dumped by pg_dump version 13.9 | ||
Using psql for Migration: | ||
Navigate to the directory containing the desired version of PostgreSQL in your terminal—the one to which you intend to migrate the data. Ensure that you have also installed the desired version of AGE as well. | ||
Execute the following command: | ||
bin/psql -d postgres-f path/to/age_dump.sql | ||
Now, it's time to check whether the data migration has been successfully executed. To do this, enter the following command in the psql terminal: | ||
SELECT * FROM cypher('test', $$ | ||
|
||
MATH (n) | ||
|
||
RETURN n | ||
|
||
$$) AS (res agtype); | ||
res | ||
\---------------------------------------------------------------------------- | ||
{"id": 1, "label": "Person", "properties": {"name": "John Doe"}}::vertex | ||
{"id": 2, "label": "Person", "properties": {"name": "Jane Smith"}}::vertex | ||
(2 rows) | ||
As you can see, the data has been successfully migrated to the new version of AGE. | ||
<!--EndFragment--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters