A Technical Guide To IPFS - The Decentralized Storage of Web3
A Technical Guide To IPFS - The Decentralized Storage of Web3
Table of Contents
What is the IPFS?
How to setup an IPFS node
How to store and retrieve IPFS content using the CLI
and HTTP
What is CID – the IPFS content-based identifier
How to reverse engineer the IPFS datastore
How to connect an IPFS node to a decentralized
network
How to exchange data using the peer-to-peer
Bitswap protocol
How to persist content from the peer-to-peer
network
"QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isap
yhCxX"
ipfs version
> ipfs version 0.8.0-rc2
Your IPFS node will generate the same CID on your local
file system as in this tutorial. That's because IPFS hashes
the content and returns its unique fingerprint, and as we
know, a secure hash function will always return the same
output given the same input.
Pin content
When you store content via the add command, IPFS will
also execute the pin command by default:
ipfs pin add QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z
Read content
The add , pin and cat commands are the most significant
IPFS functions, and you just learned them. Congrats, well
done!
CID Versions
<base><codec><hash-function><hash-length><hash-
digest>
Multihash
Multicodec
Multibase
encoding | code
base32 | b
base58btc | z
base64 | m
is CID v0
the CID string is encoded with base58btc
bafybeibkjmxftowv4lki46nad4arescoqc7kdzfnjkqux257i4j
onk44w4
CID v1
the CID string is encoded with base32
// equivalent to
ipfs cat bafybeibkjmxftowv4lki46nad4arescoqc7kdzfnjkqux257i4jonk4
> hello IPFS world by Web3Coach. BTW: Ethereum FTW
Let's dig into it and explore the default IPLD structure with
the DAG Protobuf codec.
Confused?
.
├── blocks
│ ├── 6Y
│ │ └── CIQA4XCGRCRTCCHV7XSGAZPZJOAOHLPOI6IQR3H6YQ.data
├── config
├── datastore
│ ├── 000002.ldb
│ ├── 000003.log
│ ├── CURRENT
│ ├── CURRENT.bak
│ ├── LOCK
│ ├── LOG
│ └── MANIFEST-000004
├── datastore_spec
├── keystore
└── version
ls -la hello_world.txt
> 131 bytes hello_world.txt
ls -la blocks/PV/
> 142 CIQAQIXXW2OAQSKZ6AQ2SDEYRZXWPDZNJUAFR3YORYN75I5CQ3LHPVQ.dat
vim blocks/PV/CIQA...
<8b>^A^H^B^R<83>^Ahello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs^X<83>^A
{
"Links": [],
"Data": "\b\u0002\u0012�\u0001hello IPFS world by Web3Coach. Te
}
{
"Links": [],
"Data": "\b\u0002\u0012@hello IPFS world by Web3Coach. Testing
}
{
"Links": [],
"Data": "\b\u0002\u0012@Web3Coach. Testing DAGs\nhello IPFS wor
}
{
"Links": [],
"Data": "\b\u0002\u0012\u0003AGs\u0018\u0003"
}
Data deduplication
Decentralization
Next time you want to store a file that would share part of
the content with another file, IPFS wouldn't store a
duplicate block! It would instead link to an already existing
DAG Node and only store the new, unique chunks.
Open it.
vim $IPFS_PATH/config
"Identity": {
"PeerID": "12D3KooWCBmDtsvFwDHEr...",
"PrivKey": "CAESQCj..."
},
"Bootstrap": [
"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59b...gU1ZjYZcYW3dwt",
"/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMG...UtfsmvsqQLuvuJ",
"/ip4/104.131.131.82/udp/4001/quic/p2p/Qma...UtfsmvsqQLuvuJ",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooD5...BMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2Ec...J16u19uLTa",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnM...Ucqanj75Nb"
],
ipfs daemon
I will use curl to interact with the API to keep this tutorial
"short."
{"Name":"hello_world.txt","Hash":"QmNtQtxeavDXTjCiWAAaxnhKK3LBYfF
See the official HTTP API docs for the complete list of
available commands.
How to peer with other IPFS nodes
>
/ip4/85.70.151.37/tcp/4001/p2p/QmSuCtR...aPq6h4AczBPZaoej
/ip4/91.121.168.96/udp/54001/quic/p2p/QmeC7H...8j2TQ99esS
...
...
...
Other peers can access all the content you add to your
IPFS node. There is no built-in privacy mechanism, so
never add unencrypted, sensitive/personal content to
IPFS!
http://localhost:8080/ipfs/QmT5NvUtoM5nWFfrQdVrFtvG
fKFmG7AHE8P34isapyhCxX/wiki/Anasayfa.html
peerQueues map[peer.ID]PeerQueue
pwm *peerWantManager
createPeerQueue PeerQueueFactory
ctx context.Context
psLk sync.RWMutex
sessions map[uint64]Session
peerSessions map[peer.ID]map[uint64]struct{}
self peer.ID
}
The result?
{
"Links": [
{
"Name": "0C-",
"Hash": "QmSEwJo8Z5bqVX3AhocyimJzPWetr7HgbWbwCg6zbp43AP",
"Size": 1248085
},
{
"Name": "43I",
"Hash": "QmPW3kRjncDj145bP9DVNc791FowLPwYHnqbTzfe3whdyZ",
"Size": 2611324931
},
{
"Name": "58wiki",
"Hash": "QmRNXpMRzsTHdRrKvwmWisgaojGKLPqHxzQfrXdfNkettC",
"Size": 12295304394
},
{
"Name": "92M",
"Hash": "Qmbcvk7jpBTUKdgex139Nvv7BrKocE3pQVKhNJtTU77Qv5",
"Size": 793
},
{
"Name": "A0index.html",
"Hash": "QmNqbYogAxH4mmt5WhuKN7NFEUDZ9V3Scxh7QbLwTKBJDk",
"Size": 191
}
],
"Data": "\b\u0005\u0012\u0015\u0001\u0000\u0004\u0000\u0000\u00
}
The root DAG object has five links. Four links are relatively
small, but one link points to a DAG node with a total
size of 12GB. If you inspect this DAG node, you will see
256 more links and a total cumulative (recursive) size of
12GB.
NumLinks: 256
BlockSize: 12075
LinksSize: 12034
DataSize: 41
CumulativeSize: 12295304394