Signing Files, but not quite Hashing..?
 

And it occured to me: Is there any way to tag files to tell someone that you ripped them, and they are untampered?

More like a PGP signature? (To identify the source, that is.)

(Not quite hashing. I just want to be able to rip lots of my CD's, and share them, and for someone to know that they are good, and untampered with.)

That would be a definate request of mine, for someone to know that I myself ripped it, so they know its Quality. (And maybe, search by whoever rips things by-category. Say: Music -> By: "User" or Music -> By "User".

Just a thought. Maybe someone else can explain more clearly :)

Edit: The point of the hash being to know exactly who the originator was. Like, when warez says "CLASS" on it, I know its good because it is associated with a warez group that is known for what they do, same idea here.

You know, I was just thinking the same thing today. I'd like to do the same thing with some of the DivX AVIs I make.

The first challenge here is that "By User" wouldn't reference a user name or email address, but rather a public key. Essentially you want to filter your search to those items that are signed by a key you trust. This suggests that Limewire would need a simple key management tool that lets you list the keys of content providers you trust.

As for the signatures themselves, those could be stored in Gnutella as their own items. The description line for a signature could be something like:
"|0fs73jfesa==| signed by |hjfw98\rf430|"

Where the first hash is the content hash of the data item that was signed, and the second hash is the hash of the public key of the signer. The content of this file is the signature itself. Thus, to verify the authenticity of an item, I just search for this description line to find the appropriate signature. Note that this lets multiple people sign the same item, so you can do stuff like threshold checking (e.g., I'll trust this content if it's signed by 2 out of 3 people I trust).

One problem though: if you sign the content you rip,that might make it easier for the RIAA to find you :)

Excellent :)

Any hope of this being implemented?

And if it would make it easier for the RIAA to track down, what, how many files a person has made?

All that would be needed then, is some privacy-enhancing, and a system for filtering out non-genuine Limewire, Bearshare, or whatever clients.

I personally want, what is it, the GDF? (is that the right acronym) to make a large impact on filesharing, and I hope it's not only positive, but large. This is probably my biggest request under "Make Gnutella Faster" and "Make Gnutella Secure" :D

Well the RIAA can't really do anything based on a signature. Anyone can sign any file, so the RIAA can't claim that just because you signed a file, you violated their copyright.

(begin paranoid rant)

What they could do, however, is notice that the community trusts your signature on ripped files, and from that infer that you're somehow aiding in the violation of their copyrights. I'm not a legal person, but I imagine the RIAA might go after those people that the music-sharing community trusts in this way.

Of course, this assumes the RIAA has some way of associating a public key with a person. If you use PGP, your public key is typically associated with your real name and email address, so this is certainly possible. If you're careful to only associate your public key with a pseudonym, this is less of a concern.

(end paranoid rant)

Regardless, supporting signatures on content is still a great idea for non-copyrighted material, such as free software distributions, public documents, and free artistic works.

Here's my thought.

¤ One person permanently sign a file (being the first signer), and then just have sub-signatures of users that will contain not only their signature, but a 0-10 rating of the file.

A common list of guidelines for ranking could be made for different types of media.

(Should/Could there be a limit on sub-signatures per file? 20? 50? 100?)

Signatures should not be able to be removed, of course. Too much room for tampering. Possibly be able to remove your own signatyre, but nobody elses? (there should be a minimum limit of characters of 8, and 2 numbers/symbols in the password)

I would only suppose that signatures couldnt be remvoed from files, mainly because of malicious users. Files that are rated should be averaged to attempt to weed out malicious ratings, and create a fair assesment of the file. Also, your own trusted keys could carry more weight than the public averaged rating (essentially two ratings 1/2 (1 being trusted rating, 2 being public rating)).

Thats what I got so far. I'm prietty sure theres a better way to handle this, soo, if you have an idea, shout it out.

Edit: Whoops, meant to address your issue. Adding many security enhancements to Gnutella would be needed. Some sort of IP masquerading, and encrypted file transfers between users (as i said, SecureIM style (like in trillian). More than just those things, though :)

Then, pseudonyms would keep identity hidden. (I dont have much else, mebbe I should read up on how gnutella works internally, and more info on public key exchanges, and just brush up on this in general).

top 40 project requires searching by hashes
 

We've (2 of us) been thinking about a very related thing:
- Public, non-anonymous, disucssion, ratings, top 40
done through http://one-of-us.org
- File sharing anonymous, non-secure
done through Gnutella, but we need to specify the file somehow
(like a hash.)

- notes: http://top40.one-of-us.org.
- discussion: http://one-of-us.org (search for gnutella)
Please consider checking it out and commenting on it.
I'll continue to monitor this discussion.

Tom.

NiGHTSFTP: Digital signatures can't really be "removed" by anyone -- they're just data that can be verified using a public key. The rating system you proposed has the problem that a malicious user could create a high rating for a file they like and sign it with a bunch of different keys, so it looks like many different users rating the file. It's not clear how to solve this problem.

I think the file authenticity stuff is more interesting and more tractable. It's straightforward to have GNU or RedHat publish their public key on their website and to sign their free software distributions. Users can then check the authenticity of distributions downloaded form Gnutella using those keys. As you suggested, users could even restrict their searches to match only items signed by a particular key.

A lot of work has been done on this sort of authenticated data distribution. In particular, see the
self-certifying read-only file system (SFSRO) and the cooperative file system (CFS).

Similar techniques could be applicable to content stored on Gnutella (although Gnutella can't provide the same load balancing properties as CFS).

trust network for recommending file versions
 

Sajma:
The scheme we recommend (described somewhat at http://top40.one-of-us.org) involves using a non-anonymous trust network to recommend and discuss files.

So there's 2 orthogonal components:
1) legal, non-anonymous, trusted recommendation and discussion:
Much like how open source projects post their public key or the MD5 checksum of files on their site but let you download the file from mirrors or the p2p network.
2) illegal, anonymous file sharing:
Hopefully with the added ability to let you specify the MD5 checksum of the file so that you know that it's really the same file that was highly ranked or recommended.

What about filtering the ratings?

Like: Search-> Category -> Rated 7 or better -> "Trusted Keys Only"

And you could have a small (personal) trusted key database.

Add ability to export, import, merge the key databases. (Trade em with your friends!).

NiGHTSFTP: Yes, filtering by trusted keys is exactly the way to address the problem I mentioned (malicious ratings). And sharing key databases with others you trust is also a good idea. PGP lets you do stuff like this (with key rings) and the Advogato trust network offers another approach to the same idea (see YotamAviv's earlier post about one-of-us.org).

Using a p2p storage system to store keys and ratings (more generally, certificates) also works well, since it avoids the need for a centralized or hierarchical storage system like DNS. In fact, my research is looking at just such a system.

One challenge with storing ratings as files on Gnutella is that Gnutella lookups are notoriously unreliable. That is, unless a piece of content is very popular, there's a good chance you won't find it. Therefore, it might be hard to find the ratings from people you trust, even if they exist.

How much effort would need to be made to make a system like this a reality (In LimeWire, that is)?
-=-

Would anything need to be added to "The Gnutella Protocol". Yeah, I'd suppose so.

Lets cover the basics.

* Keys sign the file.

Could this be done by appending a file itself?

...Possibly.

But that would change the SHA-1 hash after every append, defeating hashing (bad). Unless. We could add/change the ID3 tag (that might not change the hash, I should check on that..) ... But you can modify tags (facilitating the removal of keys).

There has to be something I missed. Where would signatures go?

Obviously not a central database! LOL!

* Clients want to search for the file by signer.

That means that Ultrapeers would probably need to hold this additional data, am I right?

* People want to add "Trusted Users". Thats done locally.

* People want to share trusted user database. Locally.

-=- Searching for Content
"That is, unless a piece of content is very popular, there's a good chance you won't find it." Damn freenet... err... gnutella. :P

Didnt the implementation of supernodes increase the possibility of finding files for all users?


-=- Off Topic
Has anyone else noticed that bitrates arent always displayed for MP3's in Limewire, when searching?

-=-
Also.. what about signing to prove you were the one who ripped the file? (I would like to show that "I did this", I don't care much about the RIAA. It should be an option, as far as I know). Can a key carry an "original" tag if nobody else has signed it yet. I suppose so...



(edited for: spelling, clarity)

My suggestion was to make signatures separate files. The description line for a signature file is something like SHA1(signed content)+" signed by "+SHA1(public key of signer), and the content of the signature file is the signature itself. One could generalize this to support certificates that rate files or bind names to keys.

The problem with this is that individual signatures (or certificates) are pretty unpopular items, so they might be hard to find using Gnutella. One way to fix it is to "piggyback" signature files on content files, so that they move around together. Unfortunately, if there are many signatures for a given content file, this is not practical.

Another solution is to observe that each individual only cares about singatures by the people they trust. Therefore, each person should store their own signature files (certificates). People who trust one another can also store each other's certificates. This way, a group of mutually-trusting individuals can efficiently find each another's certificates.
You can't really prove that you ripped the file and someone else didn't -- at least not without a full-blown digital rights management system (which is exactly what the RIAA wants to control the dissemination of digital music). However, you can certainly create the digital statement "I did this". Trouble is, someone else could create the same statement. The fix to this is to sign your statement with a key that other people know only belongs to you. This is the general problem that PKIs try to solve.

The basic idea is to publish your public key under a well-known name (like your email address). Your friends also sign certificates saying "Yeah, this email address is bound to that key". Other people who need to find your key look for one of these name-binding certificates signed by someone they trust (this is basically how PGP's web of trust works, though I omitted some details). You could imagine richer "digital identity systems" that bind stuff like your real name, social security number, and even digital picture to your public key. These certificates would be signed by well-known government or corporate entities, and so could be used in various secure digital applications.

Are we offtopic enough yet?
:D

How bout not even having signature files traversing the network? What about using my sharing of database idea.. but then.

SHA-1 Hashes are unique, right? So I couldnt, say, make a movie file or text file with the same hash without extreme difficulty.

Now. What bout making a database like...

|File------------------------|Signer----------|Hash------|Rating----|
|A Perfect Circle- Judith.mp3 | "6YH%%s36t^" | "Ya^h361G^@dsa" | 9
|Deftones- Magdalena.mp3 | "Ag24#61gD!c" | "243(6YHW)51d6" | 8
etc, etc, etc

Okay.

Im burnt. Maybe you can fill in what I am trying to explain. That is, not needing those little files, but using a database, with a trust system.


Umm... not till the topic turns to porn or warez!

You don't really need the signer at all. Here's what you do instead:

You can create a website like sharereactor, where people who are registered can submit new files. You set up system to let users rate the files (and calculate a rating for the user who posted them), so after a while you will have trusted users that will post files that you will trust.

Taliban:

Ratings web sites work great when the content being rated is legal (i.e. not copyrighted). The one-of-us.org site mentioned above combines content rating with a trust network. However, if users want to rate copyrighted material (like rips of music or video), a ratings website could be seen as supporting the violation of those copyrights, and thus might be shut down. Storing ratings in a decentralized system like Gnutella might be more prudent.

NiGHTSFTP:

A single database has the same problem as a single website: it's a central point of failure. Suppose we instead store copies of the ratings database on Gnutella: this helps it resist attack, but now it's unclear how to propagate updates to the database.

Perhaps what we want is a way to push the little individual ratings files around the network, and each client can grab and save the ones it cares about. So, instead of a centralized ratings database, everyone has their own local database that contains only those ratings it wants (e.g. those signed by people you trust or ratings for a particular file).

What I was thinking was, have a simple addition to the client that involves the use of trading local databases via trading them with people you know. i.e. keep a large database of files that are rated high/low, and allow the database to be sent to friends via email attachments, or what have you.

Then, "merge trusted database" function that would add to your own, creating a larger one.

The only problem with that, that I can see, is that over time the database could grow quite large, and would slow down searching, or whatever else because of its size (slowness to search the local database?).

But, how could the little signature files be propogated without flooding the network with junk?

Grr, that last post was mine. I wasnt logged in.

Storing ratings in Gnutella will be very insecure.

Taliban: In what way? Digital signatures protect the integrity of individual ratings, and if each user maintains a list of public keys that they trust, they can filter out ratings they don't care about. The main problem with storing ratings on Gnutella is that individual ones might be hard to find.

NiGHTSFTP: Searching a local database can be made pretty fast, as long as you filter out the junk. But you're right about propagating the little rating files all over the network -- that might be bad for bandwidth. A compromise solution might be to use a protocol that "syncs" two users databases. There are efficient protocols for this that use hash trees.

Any information you store in the clients and that is propagated by the clients to the rest of the network can easily be changed.

Moreover, any signature that are sent over a public network where everybody could read it, are easily spoofed. - Don't forget, that the source of many gnutella clients is open to everyone. If you know the algorithm to decrypt a signature, you can easily reverse it (reversing an encryption algorithm can mean a lot more work).

Taliban: of course "decrypting" a signature is easy to do: that's how you verify that the signature is authentic! What's important is that an attacker cannot forge a user's signature. This is only feasible if the attacker gains access to the user's private key, which is stored on the user's local machine. Without that key, forging a signature is computationally very difficult. It doesn't matter whether the attacker knows the source code -- the private key is the only thing that must be kept secret.

Whatever you can decrypt, you can also encrypt.

Your idea of one key only being able to decrypt something will not work. Current cryptographic programs work about this way:

There's key A and key B, you only need key A to encrypt but key A and key B to decrypt. So if you can decrypt something, you'll have both keys and therefore you'll be able to encrypt it also.

There is no crypto-algorithm that works the other way round.

"Data scrambled with Bob's private key can only be unscrambled with Bob's public key."

- PKI for Dummies: http://www.nwfusion.com/news/64452_05-17-1999.html

You're right, I forgot it's possible for RSA when you have a very short message.

Not just short messages.


If I encrypt, say, my database with my public key, only i could use it. This is where key management comes in.

I get my friends private key. Via AIM, email, secret meeting, whatever :)

I can then encrypt to it during export.

But why need to encrypt anything? (my method of local databases, anyway. But I'm not sure I like that. (convenience issues).

Taliban, I'm referring to standard public key cryptosystems. Each user has a private key (stored locally, so only they know it) and a public key that everyone can know about. A user signs a message using his private key. Other users can check the signature with the user's public key. There is no need for any user to ever reveal their private key to anyone.

In RSA, signatures are typically implemented by encrypting a hash of the signed file with the user's private key. To verify a signature, other users decrypt the signature block and check that it matches the hash of the signed file.

Note that actually encrypting data (such as mail messages in PGP) simply reverses this process. A sender encrypts his message with the receipient's public key. The recipient decrypts the message with her private key.

Note that in both cases, the security-sensitive operation (signing and decrypting) is done using the private key, which is never revealed to anyone.