java-victims-lib version 2

[wolferCZ]

Hello, I am attempting to create a redesigned and update version of this library.
I would like to ask for any help with the redesign.

I am bit confused with some parts of the library and would like to ask you for a help with designing its replacement.

---

I need help with setting initial goals/library requirements, and I will need someone willing to do essentially code reviews.

---

For now, I created new separate project in https://github.com/wolferCZ/victims-lib-java.

[ashcrow]

/cc @jasinner @gcmurphy

[jasinner]

Hello, I’ll help where I can. I don’t know much about the fingerprinting implementation, but can probably provide advise about API integration. Regards

…

On Wed, 28 Nov 2018 at 12:08 am, Stephen Milner ***@***.***> wrote: /cc @jasinner <https://github.com/jasinner> @gcmurphy <https://github.com/gcmurphy> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#51 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA6BdGrSnaWmy0_Ps9P-TYUYvm-HAfb1ks5uzUdggaJpZM4Y1UxK> .

[dfj]

I can help. What are the problems you are trying to solve? I think only @gcmurphy fully understands the internals of the fingerprinting mechanism, but I understand it at a conceptual level. On Wed, Nov 28, 2018 at 9:41 AM Jason Shepherd <notifications@github.com> wrote:

…

Hello, I’ll help where I can. I don’t know much about the fingerprinting implementation, but can probably provide advise about API integration. Regards On Wed, 28 Nov 2018 at 12:08 am, Stephen Milner ***@***.***> wrote: > /cc @jasinner <https://github.com/jasinner> @gcmurphy > <https://github.com/gcmurphy> > > — > You are receiving this because you were mentioned. > > > Reply to this email directly, view it on GitHub > < #51 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AA6BdGrSnaWmy0_Ps9P-TYUYvm-HAfb1ks5uzUdggaJpZM4Y1UxK > > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#51 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABAr8h_OfPmqX6GP3jXJndDv6U7vVNd7ks5uzb-DgaJpZM4Y1UxK> .

[gcmurphy]

TBH it's been so long since I looked at that code I'd pretty much be coming at it with fresh eyes anyway. From what I remember it tries to provide a consistent way of representing key aspects of the java class bytecode so that the output from two difference compilers would produce the same output when hashed.

The main entry point of that functionality is here - https://github.com/victims/victims-lib-java/blob/master/src/main/java/com/redhat/victims/fingerprint/ClassFile.java#L342

[wolferCZ]

From what I remember it tries to provide a consistent way of representing key aspects of the java class bytecode so that the output from two difference compilers would produce the same output when hashed.

... yes. It is one of the most/more essential features which has to be preserved.
Tempering with the code normalization can result in change of the result hash (fingerprint) witch can render older entries in Victims database obsolete.

However there are already some tickets regarding the code normalization (see #37).

---

Anyway normalization in ClassFile.java seemed to me a bit confusing
so I already moved it into separate util class https://github.com/wolferCZ/victims-lib-java/blob/master/src/main/java/com/redhat/victims/vlk/utils/ByteCodeNormalizer.java.
It should make creating an alternative implementations relatively easy.

[wolferCZ]

From what I gathered, the library:

• parses compiled .jar, .class, and .etc into https://github.com/victims/victims-lib-java/blob/master/src/main/java/com/redhat/victims/fingerprint/Artifact.java
  • Artifact seems to be a preprocessed representation of .jar
• flattens Artifact into "comparable" https://github.com/victims/victims-lib-java/blob/master/src/main/java/com/redhat/victims/VictimsRecord.java

After this library takes care of posting the results into Victims database.

The library also takes care of downloading, searching, and updating a private copy of Victims database ... i.e. cache.

Since than the Victims database moved to utilizing a Git repository instead of on-line service.

---

In conclusion the new version of library should offer:

• Parsing compiled code into VictimsRecord
• Comparing VictimsRecords
• Database/cache functions
  • Pulling git repository (alternatively using just path to cloned repository)
  • Searching database for given hashes

Other functionality:

• Parsing Yaml victims records into database/cache
• Serializing VictimsRecords into Yaml files (eligible for storing in victims/victims-cve-db)

[wolferCZ]

I have a question. Anybody remembers why is .jar parsed into Artifact before it can be parsed (flattened) into VictimsRecord?