Technical

Go back to the Ultimate FAQ index

<200>Technical

[201] - How do I use this software?
[202] - What are these "key types" I keep hearing about? KSK, CHK, SSK...
[203] - How is Freenet's namespace managed?
[204] - Is Freenet searchable?
[205] - Can Freenet documents be updated / deleted?
[206] - How does Freenet manage documents and disk space?
[207] - How does Freenet manage bandwidth and diskspace usage?
[208] - What prevents important documents from being discarded?
[209] - Can I just periodically reinsert/rerequest my file to ensure presence?
[210] - Can Freenet access / be accessed from the Web?
[211] - How does Freenet compare to other file sharing software?
[212] - How anonymous is Freenet?
[213] - How do I setup fred, under linux, to automatically restart if it core dumps? It is possible to run it with xinetd?
[214] - Can I setup the update.sh script to run under cron on linux (for automatic updates)? Can I do this without having to bring fred down and up?
[215] - What type of communication is used for sending routing messages/requests? TCP? UDP?
[216] - How long does a (TCP) connection last (closed immediately after sending one routhing message/request or maintained for a period of time)?
[217] - How do I control processor and bandwidth usage?

[200] - Technical Details

[201] - How do I use this software? I downloaded it, but when I run it there's no GUI.

Fred (the Freenet Reference Daemon) runs as a daemon, or service, in the background. You normally talk to it with a Freenet client. One of the services that Fred offers is called fproxy, which lets you talk to Freenet with a web browser. Point your web browser to http://127.0.0.1:8888/ for the gateway page. Try clicking the various links at the bottom of the gateway page to reach some of the popular Freenet documents.

[202] - What are these "key types" I keep hearing about? KSK, CHK, SSK...

Freenet keys come in different varieties called key types. Typically this type is included at the beginning of a key, delimited by an at- sign (@). The types are:

CHK: The Content (Cryptographic?) Hash Key is the workhorse on which other key types build. Every document has a corresponding hash value that serves as it's key. Two equal documents have equal hashes, so if different people insert the same document, this yields the same CHK. At the same time the hash function is built so that two different documents are very very unlikely to have the same hash (and therefore CHK). For all practical purposes, there is a one-to-one correspondence of CHKs to documents. All the other key types usually redirect to a CHK. So fundamentally, all real content is behind a CHK. Example: CHK@wQYjaBkpulyIBq4sbvyDL2NZ7ToPAwI,qW4hMEc6NWsx-T-etpfPAg

KSK: These are fully free-form, so can be used to tag nice, descriptive names to content. By loose convention they are seen as a hierarchic system, with levelseparateded by slashes. When you request a KSK, the Cryptographic Hash? of the keyword is computed. The CHK corresponding to this hash is retrieved, and the data pointed to by that CHK is another CHK to which you're being redirected. Example: KSK@plays/Shakespeare/Coriolanus

SSK: A secure subspace key consists of a prefix specifying the subspace, followed by a free-form name placing the key inside the subspace. Actually each subspace has two prefixes associated with it, one used for inserting, the other for retrieval of keys. The insert prefix is normally kept secret, while the retrieval prefix is given out freely. This achieves that only the holder(s) of the secret prefix may put content into the subspace, while everybody may get content out.

This kind of keys works somewhat like PGP Key pairs. You give one of them away, and the other you must keep in secret, for you will use it to insert things inside yourName Spacece" (See the next question).

[203] - How is Freenet's namespace managed?

(See previous question for more on CHK, KSK, SSK.) Central management (and therefore control) of the namespace is one of the things Freenet was designed to avoid.

CHKs don't need to be managed, every document has its CHK and can't have another one (without changing the document itself).

KSKs can be freely chosen on a first-come-first-serve basis. This may become problematic, when someone thinks it worthwhile to "squat" on all the possible combinations of eight characters. Unlike with http-URLs, though, when KSK@freenet is taken, all keys beginning with that (e.g. KSK@freenet/two) are still available.

SSKs offer everyone a private part of the namespace. The prefix is randomly generated, and of equal length, so competition for the nicest should be scarce. Somewhat like a DNS domain, the holder of a subspace has sovereignty over it.

[204] - Is Freenet searchable?

Partially. There's a gooimplementationon of a robot spider that automatically searches inside freenet, jumping from links to links, following for example indexing freesites like The Freedom Engine deep into freesites. The most used Index site of Today is DFI, which is linked from Freenets Gateway.

Check out http://giantblob.com . Beware, this is not anonymous, of course.

[205] - Can Freenet documents be updated / deleted?

Currently, a document posted to Freenet with the same name as one already present may actually serve to propagate the existing document. There is also currently no means of deleting a document from Freenet. Documents that are never requested are eventually removed through disuse.

One may employ a Date Based Redirect (DBR), though -- these are evaluated according to the current time and date. A DBR with a frequency of a day will point at a new target key every 24 hours. If this new target is always inserted before the DBR rolls over, the illusion of having the "same" document that is still updatable is achieved. If nothing resides at the current target, it appears as if the content were "deleted".

It should be noted, though, that by not going through the DBR one can access the target corresponding to any time in the past (as long as the content has not been "pushed out" to make room) or future (as long as the content has been already inserted).

The problem is it is not clear how useful they are for documents that change at irregular intervals. DBR sites, also, may not be retired, as they will brake soon, after future-dates insertions expire. At this time, DBR Sites are mainly used for frequently updated content like Dolphin's Freesite Index at "SSK@sPolFAwS69lG2Le%7eTTRzNEVarcEPAgM/DFI//"

As of today, probably the best way for most cases is the use of "edition" publishing. See the publishing section below.

[206] - How does Freenet manage documents and disk space?

Each server operator specifies an amount of storage which Freenet will reserve on his system. When Freenet's store is full, the least popular document (measured by access time) is discarded to make room. That means unless Probability Caching? is active, Freenet deletes the oldest Keys to free Space for new ones.

[207] - How does Freenet manage bandwidth and diskspace usage?

One can limit a node's bandwidth and diskspace usage. Limiting the bandwidth usage through system tweaking is recommended, but works reliably enough in newer Builds. See the corresponding section below.

[208] - What prevents important documents from being discarded?

Freenet is not intended to be an eternal archive. Because the system is completely democratic, it does not inherently distinguish between the U.N. Universal Declaration of Human Rights and my kindergarten drawings - documents are scored solely by requests. It is anticipated, however, that the current low cost of storage will make sufficent storage available on Freenet so that documents will rarely need to be discarded. Currently, at least, you can see many "DNF" (data not found) answers to freesite requests, and this is one of freenet's main problems today. Either sites are spreading slowly, or getting deleted very easily.

As a node operator, if you wish to maintain long-term Freenet storage, you should make your data store as large as you can. In addition to the obvious reason for wanting a large data store, there is a secondary reason for not using a small store. A small data store cannot cache large data chunks -- the threshold at the time of writing is about 1/200th of the size of the data store. So in order to be able to cache a 1 MB data chunk, the data store must be a bit over 200 MB. In current Freenet, this is the maximal File size. Bigger Files are split up and get FEC Encoded.

[209] - Can I just periodically reinsert/rerequest my file to ensure presence?

Yes and no. You can do it, and it will improve the odds of your file remaining present, but it does not guarantee anything. The improvement of your odds is unknowable and likely to be very small, so it's probably not worth it.

Also, the additional requests will increase load on the Freenet network, decreasing efficiency. The additional load might even be interpreted as a form of flooding or Do S? attack, to which Freenet will - rightly, in accord with its goal of foiling censorship attempts - respond in ways that defeat the original purpose. If data permanence were a goal, other methods would be both more efficient and more robust.

Inserting your file with a higher HTL (hops to live) will make it initially more available. But this does not guarantee in any way it won't be deleted later.

[210] - Can Freenet access / be accessed from the Web?

There is already one program called FCRC (available in Freenet distributions) that acts as a gateway allowing Freenet data to be accessed from the Web, and there are likely to be others. Fproxy, included with Freenet, makes Freenet available to browsers, and therefore integrates it into the Web for some people (per default, only local access is allowed).

There is not at this time (October, 2001) any facility for Freenet nodes to distribute information currently stored on the Web. Of course mirroring inserting a copy of a web page or even a whole site into Freenet is easy enough and has been done before. Documents that are stored in Freenet may also contain URLs or other references to data stored elsewhere. When viewing an in-Freenet HTML document through Fproxy links outside Freenet work transparently except for an anonymity warning.

[211] - How does Freenet compare to other file sharing software?

Superficially, Freenet will operate similarly to other information sharing systems like Napster and Gnutella. But since Freenet was designed for different goals, the underlying structure and behavior will be different. Napster, for example, relies on a single centralized server. In Gnutella searching is somewhat anonymous, but neither the publishing nor the retrieval of information is.

Freenet's decentralized, adaptive nature will make it more efficient than similar systems, more scalable, and less vulnerable to attack.

[212] - How anonymous is Freenet?

It's virtually anonymous. What you view or post on the network is very unlikely to be able to be viewed by another party (assuming they aren't looking over your sholder). If you're really worried about your anonymity, run Freenet from a cybercafe where you don't have to identify yourself. On the other hand, this approach to anonmity only works in semi-free counties, and certainly doesn't work in China anymore.

In short, Freenet is as anonymous as you can get on the Internet. It's better than www.anonymizer.com, better than freedom.net, and certainly better than using the web, but as always, USE AT YOUR OWN RISK.

If you want privacy for fetching your mp3's or porn, then you should be fine. If you really are a dissident needing to publish or retrieve information anonymously,you should wait.

See also Attacks And Weaknesses.

[213] - How do I setup Freenet, under linux, to automatically restart if it core dumps?

First log in as the user you use to start/stop freenet. Create a script file named pingFred.sh (or whatever name) in your default freenet directory and enter the following text.

#!/bin/bash

#05/25/04 - script below checks to see if there is at least

#one freenet process and if so does nothing, else it calls the

#start freenet script

let CNT=$(ps ax | grep freenet | grep -v grep | wc -l)

if [ $CNT -le 0 ]; then

echo "Restarting freenet...";

cd /usr/local/freenet/;

./start-freenet.sh;

You may need to change the 'cd /usr/local/freenet' line to the path of your freenet directory. Now save the file and do a chmod 755 pingFred.sh . Next type crontab -e and enter the following line

/5 * * * * /usr/local/freenet/pingFred.sh 1>/dev/null 2>/dev/null

Now save the file (cron entry). Basically, the script will run every 5 minutes (for different interval change the */(how many minutes to check) in the crontab entry). What it does, is see if there are any Freenet processess running and if so, it does nothing, otherwise it calls the start-freenet.sh script. Also I believe I also had to change the start-freenet.sh script to give the explicit path for the java and nice programs towards the bottom of the script. If the script does not seem to work, I suggest changing the crontab entry for the io pipe to 1>/tmp/Std Out? 2>/tmp/Std Err? and then debugging these files. You can also change the */5 to */1 to reduce the wait for cron runs, to speed up debugging. Probably, not the cleanest way to do this but it works at least.

[214] - Can I setup the update.sh script to run under cron on linux (for automatic updates)? Can I do this without having to bring fred down and up?

See the Freenet documentation about unix/Linux.

[215] - What type of communication is used for sending routing messages/requests? TCP? UDP?

Everything is sent with TCP. UDP is (currently) not used at all.

[216] - How long does a (TCP) connection last (closed immediately after sending one routhing message/request or maintained for a period of time)?

The node uses some threads (up to maxNegotiations) to opening
connections at node startup and it will keep trying to open connections up
to the open connection limit (maxNodeConnections). Idle connections are
closed after 10 minutes by default (connectionTimeout). If the node has
reached the open connection limit, it will close the connection to the worst
node if it needs to open a connection to a better node.

Because opening connections is expensive in terms of CPU usage (due to the
cryptography), only nodes that have an open connection are eglible for
routing. This will make the node's performance worse shortly after startup,
but makes routing much faster later on (because we don't have to wait for
the connection to open).

[217] - How do I control processor and bandwidth usage?

There are options to reduce bandwidth usage in the configurator.
In the config file, you'd want outputBandwidthLimit. As regards CPU,
it's harder.. one obvious possibiliy is to reduce the thread limit
(maximumThreads in the config file).

Go back to the Ultimate FAQ index