Bulk IOC Search

PacketTotal now contains almost 35,000 packet captures, over 70,000,000 records, and 20,000,000,000 data-points, this has lead to some challenges around searching and sorting the data.

Over the past few months search has gotten several improvements. We modified our algorithm to pre-process queries, and optimize them prior to search. We've doubled the number of nodes on our ElasticSearch backend, and added backend load-balancing. However, even with all these improvements making large queries, those with hundreds or even thousands of strings, poses technical issues with the current infrastructure.

Obviously, being able to search our dataset quickly is a huge priority, and to get around the technical barriers we created a separate query infrastructure on AWS. This query infrastructure provides the ability to run distributed searches asynchronously, which translates to lots of queries very fast.

To test this infrastructure we have created a tool called Bulk IOC Search Utility, which can take in a list of up to 100 line separated IPs, URLs, domains, hashes, etc. We are also planning additional projects that will leverage this infrastructure. These will be tracked here.

So give it a try! Like the rest of the site it's absolutely free. If you find the tool useful please feel free to shoot us a tweet! (@PacketTotal/@TheJaminBecker)