ekultek
|
151d44beff
|
patch for an issue where the status code was in the wrong place issue #219
|
2017-12-02 11:23:48 -06:00 |
|
ekultek
|
30ffde95c3
|
created a get_page function that does the requests for me, this way I'm not calling requests.get over and over again
|
2017-12-02 11:06:16 -06:00 |
|
ekultek
|
083e541284
|
fixes a bunch of bugs, plus some annoying output that shouldn't be showing up, including but not limited to issue #202 and issue #203
|
2017-12-01 11:05:53 -06:00 |
|
ekultek
|
b12982a958
|
patches an issue where PGP would fail (issue #179), also patches an issue where the connection refused error would not do the fix it is suppose to do (issue #180)
|
2017-11-29 09:58:16 -06:00 |
|
ekultek
|
f1e3c53cb0
|
patch for a connection refusing error (issue #177) where google will refuse your connection because you're sending to many requests to it, will sleep and try again
|
2017-11-28 10:18:07 -06:00 |
|
ekultek
|
de767965e5
|
patch for a connection refusing error (issue #177) where google will refuse your connection because you're sending to many requests to it, will sleep and try again
|
2017-11-28 10:15:28 -06:00 |
|
ekultek
|
97187c07f0
|
fixes issue #173 crawler will ignore SSL certificates, fixes #174 caches the found firewall into memory, incase we run across it again we don't waste our time trying to discover it, fixes #175 and #176 if there are unicode chars in the value it will not be saved
|
2017-11-28 08:14:47 -06:00 |
|
ekultek
|
5029005829
|
patch for an issue where a failed status code would try to create an issue, will now output the status code, the reason, and shutdown
|
2017-11-27 09:55:41 -06:00 |
|
ekultek
|
b58a6b48d4
|
enabled X-Forwarded-For headers in selenium searches
|
2017-11-26 16:44:04 -06:00 |
|
ekultek
|
d03d762ff4
|
patch for an issue where something goes wrong with the user-agent configuration during intialization of selenium, will not default to Zeus's custom agent if for any reason the user-agent is None (issue #170)
|
2017-11-26 12:02:23 -06:00 |
|
ekultek
|
46cb3d64d0
|
edited the automatic issue creator so that it will only pull the last 35 lines of the file, along with that it will now tell you the link to your issue if you have anything to add to it
|
2017-11-26 10:14:41 -06:00 |
|
ekultek
|
00da736256
|
created an interactive pause that you will be able to skip or exit from
|
2017-11-25 19:59:38 -06:00 |
|
ekultek
|
fe21f2b22b
|
moved the URL parsing to it's own class, saves some space in the searching file and is a little quicker now
|
2017-11-25 13:31:10 -06:00 |
|
ekultek
|
6def583b3a
|
patch for an issue where it will not attempt to extract the IP ban URL, will now successfully extract the URL
|
2017-11-25 13:30:06 -06:00 |
|
ekultek
|
e69e116e81
|
moved the browser setup into it's own class, saves some space in the selenium search file
|
2017-11-24 21:55:31 -06:00 |
|
ekultek
|
6f62049eb8
|
optimization for the time it takes to load the program, should be quicker now, created a search for public PGP keys will take your sites domain and search for any public PGP keys it can find, renamed some files to make more sense
|
2017-11-22 15:07:24 -06:00 |
|
ekultek
|
52af437fe4
|
moved shutdown, startup, and prompt to lib.core.common due to how much they are used
|
2017-11-20 14:27:08 -06:00 |
|
ekultek
|
9eca1950cb
|
created constants of all the log filenames, adjusted files accordingly
|
2017-11-20 14:04:11 -06:00 |
|
ekultek
|
fd4c89ffb8
|
created a common.py file and drafted write_to_log_file into it, also created an HTTP_HEADER class that will be used instead of repeating myself, fixed everything accordingly
|
2017-11-20 11:43:27 -06:00 |
|
ekultek
|
7d8727b9df
|
patch for an issue where it will claim the driver is not in your PATH, will now specify where the driver is located (issue #159)
|
2017-11-18 09:52:40 -06:00 |
|
ekultek
|
d75bb85955
|
created a Github Gist searching tool that will search 3 pages of Gists (usually around 500 links) and create a regex for the found URL, from there it will search through all the Gists and save any of them that have a match (issue #153)
|
2017-11-16 13:29:24 -06:00 |
|
ekultek
|
8b3a776af4
|
patch for an issue where the program would fail due to firefox and geckodriver not liking eachother (issue #154, issue #151), also patches an issue with the admin panel where it would write to a file no matter what happened (issue #152)
|
2017-11-16 07:34:09 -06:00 |
|
ekultek
|
25b72314e7
|
created a --show-success flag that will output the success of the dork depending on how many links have been found with it
|
2017-11-15 07:23:36 -06:00 |
|
ekultek
|
e1897912cc
|
parses the black list on multiple searches now as well, will prompt you or continue
|
2017-11-14 12:55:03 -06:00 |
|
ekultek
|
55b1285809
|
multiple tamper scripts now implemented (issue #149), checks if the status code is a 404 before saying it's XSS vulnerable (issue #147)
|
2017-11-14 11:46:14 -06:00 |
|
ekultek
|
c4af51be6d
|
created a blacklist, when a dork pulls no URLs it will add it to a blacklist, if the dork is in the blacklist and you use it again, it will prompt you if you want to continue or not
|
2017-11-13 20:45:32 -06:00 |
|
ekultek
|
d4c1e2dc1c
|
patch for an issue where the firefox profile would fail, will now specify the binary path if you hit this error (issue #143)
|
2017-11-13 09:48:02 -06:00 |
|
ekultek
|
1eb861ae16
|
patch for a reported issue (private) where if there are no URL's left it will keep going until it hits the max page, will now stop
|
2017-11-11 16:30:03 -06:00 |
|
ekultek
|
77bc6dc956
|
multiple page searching no longer requires Google's API, you can now perform as many requests as you want and get as many links as you want as long as it does not go over 500 pages. Removed a couple dependencies that needed to be there for Google's API
|
2017-11-11 15:11:06 -06:00 |
|
ekultek
|
c740eb6e74
|
full batch functionality added to the searching
|
2017-11-10 11:03:12 -06:00 |
|
ekultek
|
49cfc78d9e
|
patch for an issue where I rolled back a search setting
|
2017-11-09 08:31:29 -06:00 |
|
ekultek
|
e490a1903c
|
removed safe search enabled setting in Google API search
|
2017-11-09 08:26:50 -06:00 |
|
ekultek
|
7734f87e62
|
got rid a of a warning that never should of been there, added a comment:
|
2017-11-07 11:57:56 -06:00 |
|
ekultek
|
b28fd3eac4
|
minor edits to most information, some updates to intel scans verbosity, comment edits to sqlmap
|
2017-11-06 12:25:53 -06:00 |
|
ekultek
|
d1e4e18f80
|
minor edit to the proxy settings, you cannot run a proxy on tor, the latency would be ridiculous
|
2017-11-04 09:11:52 -05:00 |
|
ekultek
|
90ba39f3ed
|
patch for an issue where if the site refuses connection it would keep trying to crawl it (issue #134)
|
2017-11-03 09:49:58 -05:00 |
|
ekultek
|
69fd18d5a5
|
you are now able to use Tor to do the searching by passing the --tor flag, the firefox browser settings will mimic the tor browser settings and you will be able to connect via Tor, be careful, Google does not like Tor. You will also be able to connect to Tor to do the parsing
|
2017-11-02 10:27:08 -05:00 |
|
ekultek
|
bc335fa2d3
|
created a clickjacking scanner that will test a page for an X-Frame-Options header, if the header is not there, then it will create a basic HTML page with that URL as an iframe
|
2017-11-01 21:48:58 -05:00 |
|
ekultek
|
b26d29108c
|
fixes an issue where the geckodriver will not connect, usually means something happened during the installation of it, will now attempt to re-install the geckodriver (issue #132)
|
2017-11-01 10:07:15 -05:00 |
|
ekultek
|
e85c13b2b8
|
minor update to the search.py file, X-Forwarded instead of X-Forward
|
2017-10-31 14:11:05 -05:00 |
|
ekultek
|
3bc428dbd9
|
created a new flag called --x-forward that will add a header to the request called 'X-Forwarded-From' with three random IP addresses
|
2017-10-31 13:19:31 -05:00 |
|
ekultek
|
51db83833b
|
patch for a reported issue where it would fail with from a website with Unicode
|
2017-10-31 13:18:40 -05:00 |
|
ekultek
|
04cd49e722
|
added some new URL's to the skip schema, edited the DDG link to the HTML link, made sure that the URL is not just a protocol before adding to the file
|
2017-10-29 11:46:36 -05:00 |
|
ekultek
|
182e588774
|
patched a privately reported issue that would not allow you to run the blackwidow crawler behind a proxy, forgot a space in the README file
|
2017-10-28 16:53:13 -05:00 |
|
ekultek
|
f75cabb876
|
added a new requirement BeautifulSoup, fixed the crawler will pull all links with an 'a' tag and descriptor of 'href', bumped version number
|
2017-10-28 16:28:18 -05:00 |
|
ekultek
|
ecf067b6bb
|
patch for unicode errors, it appears that selenium isn't formatted to handle unicode errors (as far as I can tell) so the dork that causes the error will be skipped (issue #125)
|
2017-10-27 17:14:08 -05:00 |
|
ekultek
|
6ba7231e82
|
splitting the shell commands via shlex will make the commands more secure, and able to avoid most shell injection instances
|
2017-10-25 16:32:27 -05:00 |
|
ekultek
|
4587ec2c91
|
removed the oauth file, it is no longer needed by the program
|
2017-10-23 22:35:56 -05:00 |
|
ekultek
|
0c291d6d10
|
will now cleanup the leftover junk from being parsed, it seems that sometimes random tags get pulled with the URL
|
2017-10-23 22:27:01 -05:00 |
|
ekultek
|
d590ca22db
|
patch for an issue where when a proxy is passed without the protocol it will create an issue (issue #118), also fixes an issue with the sqlmap arguments you no longer need to pass 'true' or 'false' it will do it automatically for you (issue #123)
|
2017-10-22 19:59:31 -05:00 |
|