It has been awhile since I have written an article. In my last article, I had mentioned that I was working on developing a tool to help assist with OSINT (Open Source INTelligence) gathering on individuals. There are many times I find myself researching a threat actor that has not necessarily followed the best OPSEC, and I have partial information on the target. I am finally at the 95% completion state, and have officially named the project OSINTive (OSINT + Detective mashup). My objective when I started developing this, was to create a tool that would allow me to easily search targets based on various known criteria. I also wanted the freedom to display and export that data in various ways.
OSINTive Description
As you can see from the above screenshot, OSINTive allows you to perform reconnaissance on a target based on any of these known criteria:
- Phone Number
- Name, City, State, Country
- Username
- Email Address
Once you choose a search type, OSINTive walks you step-by-step through entering the information and finally outputs it to your terminal. You can configure OSINTive to export your results to formats such as csv, json, and basic text files. You can also configure backend criteria such as minimum probability of match, and match requirements (for example only show matches if they contain a valid phone number AND/OR email address). The tool also has the ability to accept command line arguments to automate the searching, displaying, and exporting data on targets. It also has the ability to ingest a csv file of targets with one or more pieces of known information, and bulk gathering of information.
Data Aggregation
The first few weeks I started developing OSINTive, I played around with the idea of building custom “spiders” to crawl search engines for links containing potential matches for targets, then crawl those pages for information. Though the initial proof-of-concept I developed was successful, it was slow. I also ran into issues with various search engines detecting my advanced search queries and quick traversal of links as suspicious, and blocking until a CAPTCHA was completed. I then started looking into various free and paid data aggregators. The free data aggregators I researched worked sometimes, but other times came up with completely incorrect information. I also had issues with these services randomly not being available. In the end, I found a great paid data aggregator called Pipl. Through a lot of testing (and money spent researching), I found the data to be highly accurate and abundant and that it met my needs to integrate with my tool.
As you can see from the above redacted screenshot (I ran this against targets that gave their express permission as they wanted to see what data was publicly tied to them), OSINTive outputs a wealth of information. Depending on the minimum probability and match requirements you set, OSINTive will also display potential matches.
Open Source
I have decided that I will be open sourcing OSINTive on GitHub in the near future under the GNU General Public License v3.0. This is a copyleft license, which means that derivative work must be open-source and distributed under the same license terms.
OSINTive Web
Another sister project I have been working on is a web based application version of OSINTive with enhanced functionality such as:
- User management with granular permissions.
- Ability to create “case files” that allow you to create a relationship from individual targets, and associate them to a case number.
- Ability to run new searches based on gathered results from a previous search. This comes in handy when you have a known email address, and the result shows a specific email address. A follow-up search of the email address could result in uncovering an OPSEC failure on the targets part, and show a separate associated identity/more information.
- Ability to merge case files.
As features are implemented within OSINTive core, those features will also be implemented within OSINTive web. Below is a very early screenshot of the BETA version of OSINTive web.