Merge pull request #739 from sylwit/master

Dockerize

.github/translations/README-french.md

@@ -0,0 +1,142 @@
+[![GitHub stars](https://img.shields.io/github/stars/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/stargazers)
+[![GitHub forks](https://img.shields.io/github/forks/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/network) 
+[![GitHub issues](https://img.shields.io/github/issues/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/issues) 
+[![GitHub license](https://img.shields.io/badge/license-GPL-blue.svg?style=flat-square)](https://raw.githubusercontent.com/Ekultek/Zeus-Scanner/master/.github/LICENSE.md)
+[![Twitter](https://img.shields.io/twitter/url/https/github.com/ekultek/zeus-scanner.svg?style=social)](https://twitter.com/Zeus_Scanner)
+[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://github.com/Ekultek/Zeus-Scanner#donations)
+
+# Annuaire des liens utiles
+
+- [Qu'estce que Zeus](https://github.com/Ekultek/Zeus-Scanner#zeus-scanner)
+- [Les caractéristiques de Zeus](https://github.com/Ekultek/Zeus-Scanner#features)
+- [Exigences et installation](https://github.com/Ekultek/Zeus-Scanner#requirements)
+  - [Ubuntu/Debian](https://github.com/Ekultek/Zeus-Scanner#ubuntudebian)
+  - [centOS](https://github.com/Ekultek/Zeus-Scanner#centos)
+  - [autre](https://github.com/Ekultek/Zeus-Scanner#others)
+- [Capturesécran](https://github.com/Ekultek/Zeus-Scanner#screenshots)
+- [vidéo Demo](https://vimeo.com/239885768)
+- [manuel d'utilisation](https://github.com/Ekultek/Zeus-Scanner/wiki)
+  - [Comment fonctionne Zeus](https://github.com/Ekultek/Zeus-Scanner/wiki/How-Zeus-works)
+  - [Fonctionnalité](https://github.com/Ekultek/Zeus-Scanner/wiki/Functionality)
+  - [Passant drapeaux sqlmap avec Zeus](https://github.com/Ekultek/Zeus-Scanner/wiki/Passing-flags-to-sqlmap)
+- [Informations légales](https://github.com/Ekultek/Zeus-Scanner/tree/master/.github)
+  - [Licence (GPL)](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/LICENSE.md)
+  - [Code de conduite](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CODE_OF_CONDUCT.md)
+- [Signaler un bug](https://github.com/Ekultek/Zeus-Scanner/issues/new)
+- [Ouvrir une demande de traction](https://github.com/Ekultek/Zeus-Scanner/compare)
+  - [lignes directrices de contribution](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CONTRIBUTING.md)
+- [Dons à Zeus](https://github.com/Ekultek/Zeus-Scanner#donations)
+- [Shoutouts](https://github.com/Ekultek/Zeus-Scanner#shoutouts)
+
+# Zeus-Scanner
+
+### Qu'estce que Zeus?
+
+Zeus est un utilitaire de reconnaissance avancée conçue pour rendreapplication web simple de reconnaissance. Zeus est livré avec une puissante compatibilité intégrée dansmoteur,moteur de recherche multiple analyse syntaxique URL, la capacité d'extraireURL des deux URL interdiction et WebCache, la possibilité d'exécuter plusieurs évaluations devulnérabilité sur la cible, et estmesure de contournermoteur de recherche captchas.
+
+### Caractéristiques
+
+- Un puissant construit dansmoteur d'analyse syntaxique URL
+- compatibilité des moteurs de recherche multiples (`DuckDuckGo`,` AOL`, `Bing`et`  défaut est `Google`Google`)
+- Possibilité d'extraire l'URL de l'URL d'interdiction de Google contournant ainsiblocs IP
+- Possibilité d'extraire l'URL de webcache Google
+- compatibilité proxy (`http`,` https`, `socks4`,` socks5`)
+- compatibilité proxy Tor etémulation de navigateur Tor
+- Parse `robots.txt`/`plansite .xml` et les enregistrer dans un fichier
+- évaluations devulnérabilité multiples (XSS, SQLi, clickjacking, balayageports, panneau d'administration découverte,recherches whois et plus)
+- sabotage scripts pour occultent XSS charges utiles
+- Peut fonctionner avec un agent utilisateurdéfaut personnalisé ,un des plus4000 agents-utilisateurshasard, ou un agent utilisateur personnel
+- création d'émission automatique lorsqu'une erreur inattendue survient
+- Capacité d'analyser une page Web et tirer tous les liens
+- Peut exécuter un dork singulier, dorks multiples dans un fichier donné, ou un dorkhasard dans une liste de plus5000 dorks soigneusement étudiés
+- dork listes noires lorsque passites se trouvent à la requête de recherche, va enregistrer la requête dans un fichier liste noire
+- Identifierprotection WAF / IPS / IDS de plus20 différents parefeu
+- énumération de protectiontête pour vérifier quel type de protection est assurée partêtes HTTP
+- enregistrementcookies,têtes etautres informations vitales pourfichiers journaux
+- et bien plus encore ...
+
+### Capturesécran
+
+Exécution sans options obligatoires, ouexécuter le --help` `drapeauva afficher le menu d'aide de Zeus:
+[zeus-help](https://user-images.githubusercontent.com/14183473/30176257-63391c62-93c7-11e7-94d7-68fde7818381.png)
+
+un dorkbase avec le `balayage-d`, drapeau  du dork donné lancera un navigateur automatisé et tirer le Google résultats page:
+[zeus-dork-scan](https://user-images.githubusercontent.com/14183473/30176252-618b191a-93c7-11e7-84d2-572c12994c4d.png)
+
+Appeler le `-s` drapeauvous demandera vous de démarrer le serveur API sqlmap `python sqlmapapi.py -s` de sqlmap, il va alorsconnecter à l'API et effectuer une analyse de sqlmap sur les URL trouvées.
+[zeus-sqlmap-api](https://user-images.githubusercontent.com/14183473/30176259-6657b304-93c7-11e7-81f8-0ed09a6c0268.png)
+
+Vous pouvez voir pluscapturesécran [ici](https://github.com/Ekultek/Zeus-Scanner/wiki/Screenshots)
+
+###[Demo!
+
+[![to_video](https://user-images.githubusercontent.com/14183473/31474224-feb8c022-aebe-11e7-9684-1ba83f4fd7ff..png)](https://vimeo.com/239885768)
+
+### exigences
+
+Il y a des exigences pourcela soit exécutésuccès.
+
+##### Exigencesbase
+
+- `libxml2-dev`,` libxslt1-dev`, `python-dev` sont nécessaires pour le processus d'installation
+- navigateur web Firefox est nécessairepartir de maintenant, vous aurez besoin Firefox version`<= 57 > = 51` (entre 51 et 57).fonctionnalité complète pourautres navigateurs seront ajoutées.
+- Si vous voulez exécuter sqlmaptravers vous aurez besoin d'sqlmap quelque part de l'URL sur votre système.
+- Si vous voulez exécuter un port numérisationaide nmap sur les adresses IP de l'URL. Vous aurez besoin nmap sur votre système.
+- [Geckodriver](https://github.com/mozilla/geckodriver)est nécessaire pour exécuter le navigateur Web Firefox et sera installé la première foisvous exécutez. Il sera ajouté à votre `/ usr / bin` afin qu'il puisse être exécuté dans votre ENV PATH.
+- Vous devez être `sudo` pour la première foiscoursexécutioncette façon que vous pouvez ajouter le pilote à votre PATH, vous devrez peutêtre exécutertant que`sudo` fonction de vos autorisations. _REMARQUE:_ `fonction des autorisationsvous devrez peutêtre pour toute exécution sudo impliquant le geckodriver`
+-` xvfb` est requis par `pyvirtualdisplay`,il sera installécasinstallation sur votre premier run
+
+##### package Python exigences
+
+- [sélénium WebDriver](http://www.seleniumhq.org/projects/webdriver/)paquet est nécessaire pour automatiser les appels API de navigateur Web et bypass.
+- [demandes](http://docs.python-requests.org/en/master/)paquet  est nécessaire pourconnecter à l'URL, et l'API sqlmap
+- [-nmap python](http://xael.org/pages/python-nmap-fr.html)paquet  est nécessaire pour exécuter nmap sur les adresses IP de l'URL
+- [Whichcraft](https://github.com/spookyowl/witchcraft)package est nécessaire pour vérifier si nmap et sqlmap sont sur votre système si vous voulez les utiliser
+- [pyvirtualdisplay](https://pyvirtualdisplay.readthedocs.io/en/latest/)package  est nécessaire pour masquer l'affichage du navigateur touttrouvant l'URL de recherche
+- [lxml](https://lxml.readthedocs.io/fr/latest/)est nécessaire pour analyserdonnées XML pour le plansite etenregistrertant que tel
+- [psutil](https://github.com/giampaolo/psutil)est nécessaire pour rechercherexécutionsessions API sqlmap
+- [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)est nécessaire pour tirer toutes les balises de descripteur HREF et analyser le code HTML dans une syntaxe facilement réalisable
+
+### Installation
+
+Vous pouvez télécharger le dernière [tar.gz](https://github.com/ekultek/zeus-scanner/tarball/master),le dernier [zip](https://github.com/ekultek/zeus-scanner/zipball/master),ou vous pouvez trouver le courant version stable [ici](https://github.com/Ekultek/Zeus-Scanner/releases).Sinonvous pouvez installer la dernière version de développement en suivant les instructions qui correspondentmieux à votre système d'exploitation:
+
+** _NOTE: (facultatif mais fortement conseillé)_ ** ajouter sqlmap et nmap à votre environnement PATH en les déplaçant vers `/usr/bin `ouen les ajoutant au PATH viaterminal
+
+##### Ubuntu/Debian
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev &&  git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+``` 
+ 
+##### centOS
+
+```
+sudo apt-get install gcc python-devel libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+```
+
+##### Others
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+```
+
+Celainstallera tous les Packa exigences ge ainsi que les geckodriver
+
+
+### Dons
+
+Zeus est créé par une petite équipe de développeurs qui ont une aspiration àsécurité deinformation et cherchent à réussir. Si vous aimez Zeus etvous voulez fairedon à notre financement, nous acceptons avec plaisir et appréciateur dons via:
+
+- Bitcoin (BTC): `3DAQGcAQ194NGVs16Mmv75ip45CVuE8cZy`
+- [PayPal](https://www.paypal.me/ZeusScanner)
+- Vous pouvez [Achètenous un café](https://ko-fi.com/A28355P5)
+
+vous pouvez être assuré que tousdons serviront au financementZeus pourrendre plus fiable et mieux encore, merci de l'équipe de développement Zeus
+
+### Shoutouts
+
+##### [OpenSource Projets](https://www.facebook.com/opensourceprojects/)
+
+OpenSource Projects est une page communautaire Facebook qui abut est de donnerdéveloppeurs, nouveaux et anciens, un endroit facile et simple de partager leur contributions opensource etprojets. Personnellementje pensec'est une idée géniale, je sais combien il est difficile d'obtenir votre code remarqué pargens et soutenir ces garslà100%. Allezy et leur donner un comme [ici](https://www.facebook.com/opensourceprojects/).Ils partageront tout projet opensourcevous leur envoyez gratuitement. Merci projets OpenSource pour donnerdéveloppeurs un endroit pour partagertravail avec un autre!
+

.github/translations/README-russian.md

@@ -0,0 +1,142 @@
+[![GitHub stars](https://img.shields.io/github/stars/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/stargazers)
+[![GitHub forks](https://img.shields.io/github/forks/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/network) 
+[![GitHub issues](https://img.shields.io/github/issues/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/issues) 
+[![GitHub license](https://img.shields.io/badge/license-GPL-blue.svg?style=flat-square)](https://raw.githubusercontent.com/Ekultek/Zeus-Scanner/master/.github/LICENSE.md)
+[![Twitter](https://img.shields.io/twitter/url/https/github.com/ekultek/zeus-scanner.svg?style=social)](https://twitter.com/Zeus_Scanner)
+[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://github.com/Ekultek/Zeus-Scanner#donations)
+
+# Полезные ссылки каталог
+
+- [Что такое Зевс](https://github.com/Ekultek/Zeus-Scanner#zeus-scanner)
+- [Зевса е нкции](https://github.com/Ekultek/Zeus-Scanner#features)
+- [Требования и установка](https://github.com/Ekultek/Zeus-Scanner#requirements)
+ - [Ubuntu / Debian](https://github.com/Ekultek/Zeus-Scanner#ubuntudebian)
+ - [CentOS](https://github.com/Ekultek/Zeus-Scanner#centos)
+ - [другие](https://github.com/Ekultek/Zeus-Scanner#others)
+- [Скриншоты](https://github.com/Ekultek/Zeus-Scanner#screenshots)
+- [Demo видео](https://vimeo.com/239885768)
+- [инструкцияэксплуатации](https://github.com/Ekultek/Zeus-Scanner/wiki)
+ - [Как Зевс работает](https://github.com/Ekultek/Zeus-Scanner/wiki/How-Zeus-works)
+ - [Функциональность](https://github.com/Ekultek/Zeus-Scanner/wiki/Functionality)
+ - [Передача sqlmap флаги с Зевсом](https://github.com/Ekultek/Zeus-Scanner/wiki/Passing-flags-to-sqlmap)
+- [Правовая информация](https://github.com/Ekultek/Zeus-Scanner/tree/master/.github)
+ - [License (GPL)](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/LICENSE.md)
+ - [Кодекс поведения](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CODE_OF_CONDUCT.md)
+- [Сообщить об ошибке](https://github.com/Ekultek/Zeus-Scanner/issues/new)
+- [Открыть запрос нагрузочный](https://github.com/Ekultek/Zeus-Scanner/compare)
+ - [руководящие принципы](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CONTRIBUTING.md)
+- [Пожертвования Зевса](https://github.com/Ekultek/Zeus-Scanner#donations)
+- [Shoutouts](https://github.com/Ekultek/Zeus-Scanner#shoutouts)
+
+# Zeus-сканер
+
+### Что такое Зевс?
+
+Зевс является утилитой разведки разработаночтобы сделать вебприложения разведывательный просто. Зевс поставляетсякомплекте с мощным встроенным URL разбора двигателя, множественная совместимости двигателя поиска, возможность извлечения URLадреса из обоих запрета и Webcache URLадресов, возможность запуска нескольких оценок уязвимости на цели, и может обойти каптч поисковой системы.
+
+### Особенности
+
+- мощная встроенная в URL разбора двигателя
+- Совместимость Multiple поисковой системы (`DuckDuckGo`,` AOL`, `Bing`и` Google` умолчанию является `Google`)
+- Возможность извлечения URL из запрета URLGoogle обходя таким образом IPблоки
+- Возможность извлекать из Webcache URLGoogle
+- проксисовместимость (`http`,` https`, `socks4`,` socks5`)
+- совместимостьпроксиTor и эмуляция Tor браузера
+- Разбираем `robots.txt`/`Карта сайта.xml` и сохранить их в файл
+- оценки Множественные уязвимости (XSS, SQLI, ClickJacking, сканирование портов, админка находкой, Whois поиски, и многое другое)
+- тампера скрипты запутать XSS полезных нагрузок
+- Может работать с настраиваемойумолчанию агент пользователя , один из более чем 4000 случайных пользовательских агентов или личного агента пользователя
+- Автоматическое создание проблемыкогда возникает неожиданная ошибка
+- Возможность сканировать вебстраницу и вытащить все ссылки
+- Может работать уникальный мужлан, несколько Dorks в данном файл, или случайный придурок из списка более 5000 тщательно исследовал Dorks
+- Dork черный списоккогда сайты не найдены с поисковым запросом, будет сохранить запрос в черный список файлов
+- Определение WAF / IPS / защита IDS более 20 различных брандмауэров
+- защита перечисления заголовка для проверкичто вид защиты обеспечиваетсяпомощью HTTP заголовков
+- Сохранение куки, заголовков и другая необходимая информация в логфайлы
+- и многое другое ...
+
+### Скриншоты
+
+Запуск без обязательных опций, или запустив `--help` флаг будет выводить меню помощи Зевса:
+[Zeus-помощь](https://user-images.githubusercontent.com/14183473/30176257-63391c62-93c7-11e7-94d7-68fde7818381.png)
+
+основной мужлан сканирование с `-d` флагом, из данного мужлана запустит автоматизированную браузер и тянуть Google результаты страницы:
+[Zeus-мужлан-сканирования](https://user-images.githubusercontent.com/14183473/30176252-618b191a-93c7-11e7-84d2-572c12994c4d.png)
+
+Вызов `-s` флаг запросит вы начать API сервера sqlmap `питон sqlmapapi.py -s` из sqlmap, он будет подключаться к API и выполнить sqlmap сканирование на найденный URL.
+[Zeus-sqlmap-апи](https://user-images.githubusercontent.com/14183473/30176259-6657b304-93c7-11e7-81f8-0ed09a6c0268.png)
+
+Вы можете увидеть больше скриншотов [здесь](https://github.com/Ekultek/Zeus-Scanner/wiki/Screenshots)
+
+### Demo
+
+[![to_video](https://user-images.githubusercontent.com/14183473/31474224-feb8c022-aebe-11e7-9684-1ba83f4fd7ff.png)](https://vimeo.com/239885768)
+
+### требования
+
+Есть некоторые требования для этогочтобы быть успешно работать.
+
+##### Основные требования
+
+- `libxml2-dev`,` libxslt1-dev`, `питон-dev` необходимы для процесса установки
+- веббраузер Firefox требуется как сейчас, вы будете нуждатьсяFirefox версии`<= 57 > = 51` (между 51 и 57).конечном итоге будет добавлена полная функциональность для других браузеров.
+- Если вы хотите запустить sqlmap через вам нужно будет sqlmap URLгдето в вашей системе.
+- Если вы хотите запустить сканирование портовпомощью Nmap по IPадресов URL. Вы будете нуждатьсяNmap в вашей системе.
+- [Geckodriver](https://github.com/mozilla/geckodriver)требуется для запуска веббраузера Firefox и будет установлен в первый раз при запуске. Он будет добавлен к вашему `/ USR / bin` так что он может быть запущен в вашем ENV PATH.
+- Вы должны быть `sudo` впервые работает это такчто вы можете добавить драйвер в PATH, вы можете также должны работать как`sudo` зависимости от ваших прав. _ПРИМЕЧАНИЕ:_ `зависимости от прав доступа может потребоваться быть Суда для любого бегаучастием geckodriver`
+-` xvfb` требуется на `pyvirtualdisplay`,он будет установленесли не установлен на вашемпервого запуска
+
+пакете Python##### требования
+
+- [селен WebDriver](http://www.seleniumhq.org/projects/webdriver/)пакет требуется для автоматизации веббраузер и перепускной API вызовов.
+- [запросы](http://docs.python-requests.org/en/master/)пакет требуется для подключения к URLадресу, а sqlmap API
+- [питон-птар](http://xael.org/страницы /питон-птар-en.html)пакет требуется для запуска Nmap по IPадресам URL,
+- [whichcraft](https://github.com/spookyowl/witchcraft)пакет требуетсячтобы проверитьесли птар и sqlmap находятся на вашем системыесли вы хотите использовать их
+- [pyvirtualdisplay](https://pyvirtualdisplay.readthedocs.io/en/latest/)пакет требуетсячтобы скрыть экран браузеравремя нахождения поиска URL
+- [LXML](https:// LXML .readthedocs.io / о / последние/)требуется для анализа данных XML длясайта и сохранить его как таковые
+- [psutil](https://github.com/giampaolo/psutil)требуется для поиска работы sqlmap сессий API
+- [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)требуетсячтобы вытащить все тег дескриптора HREF и разбор HTML в легко работоспособный синтаксисе
+
+### Установку
+
+Вы можете скачать последняя [tar.gz](https://github.com/ekultek/zeus-scanner/tarball/master),последняя [застежкамолния](https://github.com/ekultek/zeus-scanner/zipball/master),или вы можете найти ток стабильный релиз [здесь](https://github.com/Ekultek/Zeus-Scanner/releases).альтернативы вы можете установить последнюю версию развития, следуя инструкциикоторые наилучшимсоответствуют вашей операционной системе:
+
+** _Примечание: (обязательноно настоятельно рекомендуется)_ ** добавить sqlmap и Nmap в вашу среде PATH, перемещая их в `/ USR / бен `илипутем добавления их в PATH через терминал
+
+##### Ubuntu/Debian
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev &&  git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+``` 
+ 
+##### centOS
+
+```
+sudo apt-get install gcc python-devel libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+```
+
+##### Others
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+```
+
+Этоустановит всеM. Требования GE вместе с geckodriver
+
+
+### Пожертвования
+
+Zeus создается небольшой группой разработчиков, у которых есть стремление к информационной безопасности и стремятся добиться успеха. Если вы хотите Зевс и хотите пожертвовать наше финансирование, мырадостью и благодарностью принимаем пожертвование через:
+
+- Bitcoin (BTC): `3DAQGcAQ194NGVs16Mmv75ip45CVuE8cZy`
+- [PayPal](https://www.paypal.me/ZeusScanner)
+- Или вы можете [купить нам кофе](https://ko-fi.com/A28355P5)
+
+Вы можете быть уверенычто все пожертвования пойдут на финансирование Зевсачтобы сделать его более надежным и даже лучше, спасибо от команды разработчиков Zeus
+
+### Shoutouts
+
+##### [OpenSource проекты](https://www.facebook.com/opensourceprojects/)
+
+OpenSource проекты это страница Facebook сообществакто цель состоитчтобы дать разработчикам, новые и старые, а легко и просто месточтобы разделить их вклад OpenSource и проекты. Я лично считаюэто огромная идея, я знаюкак трудно получить код заметил людьми и поддерживает эти ребята100%. Идите вперед и дать им как [здесь](https://www.facebook.com/opensourceprojects/).Они будут делиться любой проектоткрытым исходным кодом вы отправить их бесплатно. Спасибо OpenSource проектов для предоставления разработчикам место для обмена работу друг с другом!
+

.github/translations/README-spanish.md

@@ -0,0 +1,139 @@
+[![GitHub stars](https://img.shields.io/github/stars/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/stargazers)
+[![GitHub forks](https://img.shields.io/github/forks/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/network) 
+[![GitHub issues](https://img.shields.io/github/issues/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/issues) 
+[![GitHub license](https://img.shields.io/badge/license-GPL-blue.svg?style=flat-square)](https://raw.githubusercontent.com/Ekultek/Zeus-Scanner/master/.github/LICENSE.md)
+[![Twitter](https://img.shields.io/twitter/url/https/github.com/ekultek/zeus-scanner.svg?style=social)](https://twitter.com/Zeus_Scanner)
+[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://github.com/Ekultek/Zeus-Scanner#donations)
+
+# Directorio de enlaces útiles
+
+- [Qué es Zeus](https://github.com/Ekultek/Zeus-Scanner#zeus-scanner)
+- [Funciones de Zeus](https://github.com/Ekultek/Zeus-Scanner#features)
+- [Requisitos e instalación](https://github.com/Ekultek/Zeus-Scanner#requirements)
+  - [Ubuntu/Debian](https://github.com/Ekultek/Zeus-Scanner#ubuntudebian)
+  - [centOS](https://github.com/Ekultek/Zeus-Scanner#centos)
+  - [otro](https://github.com/Ekultek/Zeus-Scanner#others)
+- [Capturas de pantalla](https://github.com/Ekultek/Zeus-Scanner#screenshots)
+- [Video de demostración](https://vimeo.com/239885768)
+- [Manual de usuario](https://github.com/Ekultek/Zeus-Scanner/wiki)
+  - [Cómo funciona Zeus](https://github.com/Ekultek/Zeus-Scanner/wiki/How-Zeus-works)
+  - [Funcionalidad](https://github.com/Ekultek/Zeus-Scanner/wiki/Functionality)
+  - [Pasando banderas sqlmap con Zeus](https://github.com/Ekultek/Zeus-Scanner/wiki/Passing-flags-to-sqlmap)
+- [Información legal](https://github.com/Ekultek/Zeus-Scanner/tree/master/.github)
+  - [Licencia (GPL)](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/LICENSE.md)
+  - [Código de conducta](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CODE_OF_CONDUCT.md)
+- [Informar de un error](https://github.com/Ekultek/Zeus-Scanner/issues/new)
+- [Abrir solicitud de extracción](https://github.com/Ekultek/Zeus-Scanner/compare)
+  - [Directrices de contribución](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/CONTRIBUTING.md)
+- [Donaciones a Zeus](https://github.com/Ekultek/Zeus-Scanner#donations)
+- [Shoutouts](https://github.com/Ekultek/Zeus-Scanner#shoutouts)
+
+# Zeus-Scanner
+
+### ¿Qué es Zeus?
+
+Zeus es una utilidad de reconocimiento avanzada diseñada para hacer que el reconocimiento de aplicaciones web sea simple. Zeus viene completo con un poderoso motor de análisis integrado de URL, compatibilidad con múltiples motores de búsqueda, la capacidad de extraer URL de las URL de prohibición y de caché web, la capacidad de ejecutar múltiples evaluaciones de vulnerabilidad en el objetivo y puede eludir los captchas de los motores de búsqueda.
+
+### Caracteristicas
+
+ - Un potente motor de análisis de URL incorporado
+ - La compatibilidad con múltiples motores de búsqueda (`DuckDuckGo`,` AOL`, `Bing` y` Google` por defecto es `Google`
+ - Posibilidad de extraer la URL de la URL de prohibición de Google evitando así los bloques de IP
+ - Posibilidad de extraer de la URL de caché web de Google
+ - Compatibilidad proxy (`http`,` https`, `socks4`,` socks5`
+ - Compatibilidad Tor proxy y emulación de navegador Tor
+ - Parse `robots.txt` /` sitemap.xml` y guárdelos en un archivo
+ - Múltiples evaluaciones de vulnerabilidad (XSS, SQLi, clickjacking, escaneo de puertos, hallazgos de panel de administración, búsquedas de whois, y más)
+ - Guiones de sabotaje para ofuscar cargas útiles XSS
+ - Se puede ejecutar con un agente de usuario predeterminado personalizado, uno de los más de 4000 agentes de usuario aleatorios o un agente de usuario personal
+ - Creación automática de problemas cuando surge un error inesperado
+ - Posibilidad de rastrear una página web y extraer todos los enlaces
+ - Puede ejecutar un dork singular, múltiples dorks en un archivo determinado, o un dork aleatorio de una lista de más de 5000 dorks cuidadosamente investigados
+ - Lista negra de Dork cuando no se encuentran sitios con la consulta de búsqueda, guardará la consulta en un archivo de lista negra
+ - Identificar la protección WAF / IPS / IDS de más de 20 firewalls diferentes
+ - Enumeración de protección de encabezado para verificar qué tipo de protección se proporciona a través de encabezados HTTP
+ - Guardar cookies, encabezados y otra información vital para registrar archivos
+ - y mucho más...
+
+### Capturas de pantalla
+
+Si ejecuta sin opciones obligatorias o si ejecuta el indicador `--help`, se mostrará el menú de ayuda de Zeus:
+! [zeus-help](https://user-images.githubusercontent.com/14183473/30176257-63391c62-93c7-11e7-94d7-68fde7818381.png)
+Un escaneo de dork básico con la bandera `-d`, del dork dado lanzará un navegador automatizado y extraerá los resultados de la página de Google:
+! [zeus-dork-scan](https://user-images.githubusercontent.com/14183473/30176252-618b191a-93c7-11e7-84d2-572c12994c4d.png)
+Llamar al indicador `-s` le pedirá que inicie el servidor de la API sqlmap` python sqlmapapi.py -s` desde sqlmap, luego se conectará a la API y realizará un análisis de sqlmap en la URL encontrada.
+! [zeus-sqlmap-api](https://user-images.githubusercontent.com/14183473/30176259-6657b304-93c7-11e7-81f8-0ed09a6c0268.png)
+
+Puede ver más capturas de pantalla [aquí](https://github.com/Ekultek/Zeus-Scanner/wiki/Screenshots)
+
+### Demo
+
+[![to_video](https://user-images.githubusercontent.com/14183473/31474224-feb8c022-aebe-11e7-9684-1ba83f4fd7ff.png)
+](https://vimeo.com/239885768)
+
+### Requisitos
+
+Hay algunos requisitos para que esto se ejecute con éxito.
+
+##### Requerimientos básicos
+
+ - `libxml2-dev`,` libxslt1-dev`, `python-dev` son necesarios para el proceso de instalación
+ - Se requiere navegador web Firefox a partir de ahora, necesitarás la versión de Firefox `<= 57> = 51` (entre 51 y 57). Se agregará la funcionalidad completa para otros navegadores.
+ - Si desea ejecutar sqlmap a través de la URL, necesitará sqlmap en algún lugar de su sistema.
+ - Si desea ejecutar un escaneo de puertos usando nmap en las direcciones IP de la URL. Necesitarás nmap en tu sistema.
+ - [Geckodriver](https://github.com/mozilla/geckodriver) es necesario para ejecutar el navegador web firefox y se instalará la primera vez que ejecute. Se agregará a su `/ usr / bin` para que pueda ejecutarse en su ENV PATH.
+ - Debe ser `sudo` por primera vez ejecutando esto para que pueda agregar el controlador a su RUTA, también puede necesitar ejecutar como` sudo` dependiendo de sus permisos. _NOTA: _`Dependiendo de los permisos, puede que necesite sudo para cualquier ejecución que involucre al geckodriver`
+ - `xvfb` es requerido por` pyvirtualdisplay`, se instalará si no está instalado en su primera ejecución
+ 
+##### Requisitos del paquete de Python
+
+ - Se requiere el paquete [selenium-webdriver](http://www.seleniumhq.org/projects/webdriver/) para automatizar el navegador web y eludir las llamadas API.
+ - Se requiere el paquete [requests](http://docs.python-requests.org/en/master/) para conectarse a la URL y a la API de sqlmap.
+ - Se requiere el paquete [python-nmap](http://xael.org/pages/python-nmap-en.html) para ejecutar nmap en las direcciones IP de la URL
+ - El paquete [witchcraft](https://github.com/spookyowl/witchcraft) es necesario para verificar si nmap y sqlmap están en su sistema si desea usarlos
+ - Se requiere el paquete [pyvirtualdisplay](https://pyvirtualdisplay.readthedocs.io/en/latest/) para ocultar la visualización del navegador mientras se encuentra la URL de búsqueda
+ - [lxml](https://lxml.readthedocs.io/en/latest/) es necesario para analizar los datos XML del mapa del sitio y guardarlo como tal
+ - [psutil](https://github.com/giampaolo/psutil) es necesario para buscar ejecutar sesiones API de sqlmap
+ - [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) es necesario para extraer todas las etiquetas de descriptor HREF y analizar el HTML en una sintaxis fácil de usar
+ 
+### Instalación
+
+Puede descargar la última [tar.gz](https://github.com/ekultek/zeus-scanner/tarball/master), la última [zip](https://github.com/ekultek/zeus-scanner/zipball/master), o puede encontrar la versión estable actual [aquí](https://github.com/Ekultek/Zeus-Scanner/releases). Alternativamente, puede instalar la última versión de desarrollo siguiendo las instrucciones que mejor se adapten a su sistema operativo:
+
+**_NOTA: (opcional pero muy recomendable)_ ** agregue sqlmap y nmap a su RUTA del entorno moviéndolos a `/usr/bin` o agregándolos a la RUTA a través de la terminal
+
+##### Ubuntu / Debian
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev && git clon https://github.com/ekultek/zeus-scanner.git y& cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus .py
+```
+ 
+##### centOS
+
+```
+sudo apt-get install gcc python-devel libxml2-dev libxslt1-dev python-dev && git clon https://github.com/ekultek/zeus-scanner.git y& cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
+```
+
+##### Otros
+
+```
+sudo apt-get install libxml2-dev libxslt1-dev python-dev && git clon https://github.com/ekultek/zeus-scanner.git y& cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus .py
+```
+
+Esto instalará todos los requisitos del paquete junto con el geckodriver
+
+### Donaciones
+
+Zeus es creado por un pequeño equipo de desarrolladores que aspiran a la seguridad de la información y se esfuerzan por tener éxito. Si te gusta Zeus y quieres donar a nuestra financiación, agradecemos y agradecemos las donaciones a través de:
+
+ - Bitcoin (BTC): `3DAQGcAQ194NGVs16Mmv75ip45CVuE8cZy`
+ - [PayPal](https://www.paypal.me/ZeusScanner)
+ - O puedes [Cómpranos un café](https://ko-fi.com/A28355P5)
+ 
+Puede estar seguro de que todas las donaciones se destinarán a la financiación de Zeus para que sea más confiable e incluso mejor, gracias del equipo de desarrollo de Zeus.
+
+### Shoutsouts
+
+##### [Proyectos de OpenSource](https://www.facebook.com/opensourceprojects/)
+
+OpenSource Projects es una página de la comunidad de Facebook cuyo objetivo es brindar a los desarrolladores, nuevos y antiguos, un lugar fácil y simple para compartir sus contribuciones y proyectos de código abierto. Personalmente creo que esta es una idea increíble, sé lo difícil que es hacer que la gente note su código y apoyar a estos tipos al 100%. Continúa y dales un me gusta [aquí](https://www.facebook.com/opensourceprojects/). Compartirán cualquier proyecto de código abierto que les envíe de forma gratuita. ¡Gracias OpenSource Projects por darles a los desarrolladores un lugar para compartir el trabajo entre ellos!

.gitignore

@@ -1,3 +1,4 @@
+test.py
 log/
 geckodriver.log
 *.pyc

Dockerfile

@@ -0,0 +1,34 @@
+FROM ubuntu:18.10
+
+WORKDIR /app
+
+RUN apt update && \
+	apt install -y \
+	libxml2-dev \
+	libxslt1-dev \
+	libgtk-3-dev \
+	libdbus-glib-1-2 \
+	python-dev \ 
+	python-pip \
+	git \
+	curl \
+	nmap \
+	sqlmap \ 
+	xvfb \
+	&& rm -rf /var/lib/apt/lists/* 
+
+ARG GECKO_DRIVER_VERSION=0.23.0
+ARG FIREFOX_VERSION=58.0.2
+
+RUN git clone https://github.com/ekultek/zeus-scanner.git . && \
+	pip install -r requirements.txt 
+
+RUN curl -L https://github.com/mozilla/geckodriver/releases/download/v${GECKO_DRIVER_VERSION}/geckodriver-v${GECKO_DRIVER_VERSION}-linux64.tar.gz | tar xz -C /usr/bin 
+
+RUN curl -L https://ftp.mozilla.org/pub/firefox/releases/${FIREFOX_VERSION}/linux-$(uname -m)/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -o firefox.tar.bz2 && \
+	tar xjf firefox.tar.bz2 -C /opt && \
+	rm firefox.tar.bz2 && \
+	ln -s /opt/firefox/firefox /usr/bin/firefox
+
+CMD ["python", "zeus.py"]
+

README.md

@@ -1,17 +1,23 @@
+**_NOTE: due to dumbass people, automatic issue creation has been turned off until further notice_**
+
+----
+
 [![GitHub stars](https://img.shields.io/github/stars/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/stargazers)
 [![GitHub forks](https://img.shields.io/github/forks/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/network) 
 [![GitHub issues](https://img.shields.io/github/issues/ekultek/zeus-scanner.svg?style=flat-square)](https://github.com/ekultek/zeus-scanner/issues) 
 [![GitHub license](https://img.shields.io/badge/license-GPL-blue.svg?style=flat-square)](https://raw.githubusercontent.com/Ekultek/Zeus-Scanner/master/.github/LICENSE.md)
-[![Twitter](https://img.shields.io/twitter/url/https/github.com/ekultek/zeus-scanner.svg?style=social)](https://twitter.com/Zeus_Scanner)
+[![Twitter](https://img.shields.io/twitter/url/https/github.com/ekultek/zeus-scanner.svg?style=social)](https://twitter.com/stay__salty)
 [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://github.com/Ekultek/Zeus-Scanner#donations)
 
 # Helpful links directory
 
+- [Translations](https://github.com/Ekultek/Zeus-Scanner#translations)
 - [What is Zeus](https://github.com/Ekultek/Zeus-Scanner#zeus-scanner)
 - [Zeus's features](https://github.com/Ekultek/Zeus-Scanner#features)
 - [Requirements and installation](https://github.com/Ekultek/Zeus-Scanner#requirements)
   - [Ubuntu/Debian](https://github.com/Ekultek/Zeus-Scanner#ubuntudebian)
   - [centOS](https://github.com/Ekultek/Zeus-Scanner#centos)
+  - [Backbox](https://github.com/Ekultek/Zeus-Scanner#backbox)
   - [other](https://github.com/Ekultek/Zeus-Scanner#others)
 - [Screenshots](https://github.com/Ekultek/Zeus-Scanner#screenshots)
 - [Demo video](https://vimeo.com/239885768)
@@ -78,7 +84,7 @@ There are some requirements for this to be run successfully.
 ##### Basic requirements
 
  - `libxml2-dev`, `libxslt1-dev`, `python-dev` are required for the installation process
- - Firefox web browser is required as of now, you will need Firefox version `<=57 >=51` (between 51 and 57). Full functionality for other browsers will eventually be added.
+ - Firefox web browser is required as of now, you will need Firefox version `<=58 >=52` (between 52 and 58). Full functionality for other browsers will eventually be added.
  - If you want to run sqlmap through the URL's you will need sqlmap somewhere on your system.
  - If you want to run a port scan using nmap on the URL's IP addresses. You will need nmap on your system.
  - [Geckodriver](https://github.com/mozilla/geckodriver) is required to run the firefox web browser and will be installed the first time you run. It will be added to your `/usr/bin` so that it can be run in your ENV PATH.
@@ -98,7 +104,7 @@ There are some requirements for this to be run successfully.
 
 ### Installation
 
-You can download the latest [tar.gz](https://github.com/ekultek/zeus-scanner/tarball/master), the latest [zip](https://github.com/ekultek/zeus-scanner/zipball/master), or you can find the current stable release [here](https://github.com/Ekultek/Zeus-Scanner/releases/tag/v1.3). Alternatively you can install the latest development version by following the instructions that best match your operating system:
+You can download the latest [tar.gz](https://github.com/ekultek/zeus-scanner/tarball/master), the latest [zip](https://github.com/ekultek/zeus-scanner/zipball/master), or you can find the current stable release [here](https://github.com/Ekultek/Zeus-Scanner/releases/tag/v1.5). Alternatively you can install the latest development version by following the instructions that best match your operating system:
 
 **_NOTE: (optional but highly advised)_** add sqlmap and nmap to your environment PATH by moving them to `/usr/bin` or by adding them to the PATH via terminal
 
@@ -114,6 +120,36 @@ sudo apt-get install libxml2-dev libxslt1-dev python-dev &&  git clone https://g
 sudo apt-get install gcc python-devel libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
 ```
 
+#### Backbox
+
+64 bit installation:
+```
+sudo -s << EOF
+aptitude purge firefox
+wget https://ftp.mozilla.org/pub/firefox/releases/57.0/linux-x86_64/en-US/firefox-57.0.tar.bz2
+tar -xjf firefox-57.0.tar.bz2
+rm -rf /opt/firefox*
+mv firefox /opt/firefox57
+mv /usr/bin/firefox /usr/bin/firefoxold
+ln -s /opt/firefox57/firefox-bin /usr/bin/firefox
+apt-get install libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && pip2 install -r requirements.txt && python zeus.py
+EOF
+```
+
+32 bit installation:
+```
+sudo -s << EOF
+aptitude purge firefox
+wget https://ftp.mozilla.org/pub/firefox/releases/57.0/linux-i686/en-US/firefox-57.0.tar.bz2
+tar -xjf firefox-57.0.tar.bz2
+rm -rf /opt/firefox*
+mv firefox /opt/firefox57
+mv /usr/bin/firefox /usr/bin/firefoxold
+ln -s /opt/firefox57/firefox-bin /usr/bin/firefox
+apt-get install libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && pip2 install -r requirements.txt && python zeus.py
+EOF
+```
+
 ##### Others
 
 ```
@@ -138,3 +174,10 @@ You can be assured that all donations will go towards Zeus funding to make it mo
 ##### [OpenSource Projects](https://www.facebook.com/opensourceprojects/)
 
 OpenSource Projects is a Facebook community page who's goal is to give developers, new and old, a easy and simple place to share their opensource contributions and projects. I personally think this is an awesome idea, I know how hard it is to get your code noticed by people and support these guys 100%. Go ahead and give them a like [here](https://www.facebook.com/opensourceprojects/). They will share any opensource project you send them for free. Thank you OpenSource Projects for giving developers a place to share work with one another!
+
+
+### Translations
+ 
+ - [Spanish](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/translations/README-spanish.md)
+ - [Russian](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/translations/README-russian.md)
+ - [French](https://github.com/Ekultek/Zeus-Scanner/blob/master/.github/translations/README-french.md)

bin/unzip_gecko.py

@@ -1,5 +1,9 @@
 import os
+import sys
+import time
+import shlex
 import platform
+import threading
 import subprocess
 import tarfile
 try:
@@ -13,6 +17,27 @@ import lib.core.common
 import lib.core.settings
 
 
+stop_animation = False
+xvfb_path = "{}/etc/scripts/install_xvfb.sh".format(os.getcwd())
+
+
+def animation(text):
+    global stop_animation
+    i = 0
+    while not stop_animation:
+        temp_text = list(text)
+        if i >= len(temp_text):
+            i = 0
+        temp_text[i] = temp_text[i].upper()
+        temp_text = ''.join(temp_text)
+        sys.stdout.write("\033[92m{}\r\033[0m".format(temp_text))
+        sys.stdout.flush()
+        i += 1
+        time.sleep(0.1)
+    else:
+        pass
+
+
 def disclaimer():
     question = raw_input(
         "\033[91mAttacking targets without consent is not only illegal, but it "
@@ -28,7 +53,7 @@ def disclaimer():
     else:
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
             "you have not agreed with the terms of service, so "
-            "Zeus will shut down now...", level=50
+            "Zeus will shut down now", level=50
         ))
         return False
 
@@ -46,6 +71,8 @@ def parse_hosts(filepath="/etc/hosts"):
 
 
 def find_tools(to_search=("sqlmap", "nmap"), directory="{}/bin/paths", filename="path_config.ini"):
+    global stop_animation
+
     lib.core.settings.create_dir(directory.format(os.getcwd()))
     full_path = "{}/{}".format(
         directory.format(os.getcwd()),
@@ -62,6 +89,8 @@ def find_tools(to_search=("sqlmap", "nmap"), directory="{}/bin/paths", filename=
             path_schema[item] = None
     for key, value in path_schema.iteritems():
         if value is None:
+            stop_animation = True
+            print("\n")
             provided_path = lib.core.common.prompt(
                 "what is the full path to {} on your system".format(key)
             )
@@ -78,9 +107,9 @@ def config_gecko_version(browser_version):
     figure out which gecko version you need
     """
     version_specs = {
-        (56, 57): 19,
+        (57, 58): 19,
-        (55, 54): 18,
+        (56, 55, 54): 18,
-        (53, 52, 51): 17
+        (53, 52): 17
     }
     if isinstance(browser_version, (tuple, list, set)):
         major = browser_version[0]
@@ -112,11 +141,13 @@ def check_xvfb(exc="Xvfb"):
     """
     test for xvfb on the users system
     """
+    global xvfb_path
+    global stop_animation
     if whichcraft.which(exc) is None:
-        lib.core.settings.logger.info(lib.core.settings.set_color(
+        cmd = shlex.split("sudo sh {}".format(xvfb_path))
-            "installing Xvfb, required by pyvirutaldisplay..."
+        subprocess.call(cmd)
-        ))
+        stop_animation = True
-        subprocess.call(["sudo", "apt-get", "install", "xvfb"])
+
     else:
         return True
 
@@ -136,53 +167,45 @@ def check_if_run(file_check="{}/bin/executed.txt"):
             return True
 
 
-def untar_gecko(filename="{}/bin/drivers/geckodriver-v0.{}.0-linux{}.tar.gz", verbose=False):
+def untar_gecko(filename="{}/bin/drivers/geckodriver-v0.{}.0-linux{}.tar.gz"):
     """
     untar the correct gecko driver for your computer architecture
     """
+    global stop_animation
+
     arch_info = {"64bit": "64", "32bit": "32"}
     file_arch = arch_info[platform.architecture()[0]]
-    ff_version = lib.core.settings.get_browser_version()
+    ff_version = lib.core.settings.get_browser_version(output=False)
     if isinstance(ff_version, str) or ff_version is None:
+        stop_animation = True
         ff_version = lib.core.common.prompt(
-            "enter your firefox browser version (if you don't know it run firefox --version"
+            "enter your firefox browser version (if you don't know it run firefox --version)"
         )
     gecko_version = config_gecko_version(ff_version)
     if gecko_version is None:
+        stop_animation = True
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
-            "your current firefox version is not supported by Zeus...", level=50
+            "your current firefox version is not supported by Zeus", level=50
         ))
         lib.core.common.shutdown()
     gecko_full_filename = filename.format(os.getcwd(), gecko_version, file_arch)
     with open(lib.core.settings.GECKO_VERSION_INFO_PATH, "a+") as log:
         log.write(gecko_full_filename.split("/")[-1])
     tar = tarfile.open(filename.format(os.getcwd(), gecko_version, file_arch), "r:gz")
-    if verbose:
-        lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "extracting the correct driver for your architecture '{}...", level=10
-        ))
     try:
         tar.extractall("/usr/bin")
-        if verbose:
-            lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "driver extracted into /usr/bin (you may change this, but ensure that it "
-                "is in your PATH)...", level=10
-            ))
     except IOError as e:
         if "Text file busy" in str(e):
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "the driver is already installed..."
-            ))
             tar.close()
             pass
     except Exception as e:
         if "[Errno 13] Permission denied: '/usr/bin/geckodriver'" in str(e):
             lib.core.settings.logger.exception(lib.core.settings.set_color(
-                "first run must be ran as root (sudo python zeus.py)...", level=50
+                "first run must be ran as root (sudo python zeus.py)", level=50
             ))
         else:
             lib.core.settings.logger.exception(lib.core.settings.set_color(
-                "ran into exception '{}', logged to current log file...".format(e), level=50
+                "ran into exception '{}', logged to current log file".format(e), level=50
             ))
         exit(-1)
     tar.close()
@@ -192,10 +215,6 @@ def ensure_placed(item="geckodriver", verbose=False):
     """
     use whichcraft to ensure that the driver has been placed in your PATH variable
     """
-    if verbose:
-        lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "ensuring that the driver exists in your system path...", level=10
-        ))
     if not whichcraft.which(item):
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
             "the executable '{}' does not appear to be in your /usr/bin PATH. "
@@ -204,10 +223,6 @@ def ensure_placed(item="geckodriver", verbose=False):
         ))
         exit(-1)
     else:
-        if verbose:
-            lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "driver exists, continuing...", level=10
-            ))
         return True
 
 
@@ -215,39 +230,28 @@ def main(rewrite="{}/bin/executed.txt", verbose=False):
     """
     main method
     """
-    if verbose:
-        lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "verifying operating system...", level=10
-        ))
     if not check_os():
         raise NotImplementedError(lib.core.settings.set_color(
             "as of now, Zeus requires Linux to run successfully "
             "your current operating system '{}' is not implemented "
-            "yet...".format(platform.platform()), level=50
+            "yet".format(platform.platform()), level=50
         ))
     if check_if_run():
         if not disclaimer():
             exit(1)
-        lib.core.settings.logger.info(lib.core.settings.set_color(
+        t = threading.Thread(target=animation, args=(
-            "seems this is your first time running the application, "
+            "seems this is your first time running the application, doing setup please wait..",))
-            "doing setup please wait..."
+        t.daemon = True
-        ))
+        t.start()
-        if verbose:
-            lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "checking if xvfb is on your system...", level=10
-            ))
         find_tools()
         check_xvfb()
-        untar_gecko(verbose=verbose)
+        untar_gecko()
         parse_hosts()
         if ensure_placed(verbose=verbose):
             with open(rewrite.format(os.getcwd()), "w") as rw:
                 rw.write("TRUE")
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "done, continuing process..."
+            "done, continuing process"
         ))
     else:
-        if verbose:
+        pass
-            lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "already ran, skipping...", level=10
-            ))

etc/checksum/md5sum.md5

@@ -1,84 +1,119 @@
-e4ea2d20dd1e0ec58e68159689e2cb74  ./zeus.py
+685a20fa3fc7652b5c3e39821cdc6f25  ./zeus.py
 4b32db388e8acda35570c734d27c950c  ./etc/scripts/launch_sqlmap.sh
 6ad5f22ec4a6f8324bfb1b01ab6d51ec  ./etc/scripts/cleanup.sh
-74d7bee13890a9dd279bb857591647ce  ./etc/scripts/reinstall.sh
+869025acb457dc881e53e440aa11dd7b  ./etc/scripts/reinstall.sh
 155c9482f690f1482f324a7ffd8b8098  ./etc/scripts/fix_pie.sh
 0e435c641bc636ac0b3d54e032d9cf6a  ./etc/scripts/install_nmap.sh
+440431165b2db8a537c1a93cb2232f16  ./etc/scripts/install_xvfb.sh
 66b11aa388ea909de7b212341259a318  ./etc/auths/git_auth
 8f686b05c5c5dfc02f0fcaa7ebc8677c  ./etc/auths/whois_auth
 d3ad89703575a712a0aeead2b176d8c5  ./etc/html/clickjacking_test_page.html
 642a77905d8bb4e5533e0e9c2137c0fa  ./etc/text_files/agents.txt
 82cc68f46539d0255f7ce14cd86cd49b  ./etc/text_files/link_ext.txt
-b4ac1dab7fee43d1295b0d940e11fac1  ./etc/text_files/dorks.txt
+c57ac34fe965961917ac8a207df256d5  ./etc/text_files/dorks.txt
-dc7bfc3d7b9b23340ee37806316bd770  ./etc/text_files/xss_payloads.txt
+cf85d83da34d70720193d83950c31fdc  ./etc/text_files/xss_payloads.txt
-b795ccb193f307a8e9a4a888c018241a  ./etc/xml/headers.xml
+6cabeb9919d2301efc4ba3d8869282d6  ./etc/checksum/md5sum.md5
+5250f0aa13b8af4775efa506e77de1ce  ./etc/xml/headers.xml
 d41d8cd98f00b204e9800998ecf8427e  ./bin/__init__.py
-19f799120461d5f30b17ba39e8a92598  ./bin/unzip_gecko.py
+3be7ee6f4267e0d0cf2143b58792527b  ./bin/paths/path_config.ini
+fa5084cc7ee56ff2df8631b76be5be4d  ./bin/unzip_gecko.py
+c0d83f0b82a6b30de8811e69e6d95c61  ./bin/executed.txt
 dc1eb4ebe0f372af48b5a9c107ebc68d  ./bin/drivers/geckodriver-v0.18.0-linux32.tar.gz
 be18faeea6e7db9db6990d8667e2298f  ./bin/drivers/geckodriver-v0.17.0-linux64.tar.gz
 79b1a158f96d29942a111c0905f1c807  ./bin/drivers/geckodriver-v0.17.0-linux32.tar.gz
 ca6935a72fd0527d15a78a17a35e56e8  ./bin/drivers/geckodriver-v0.19.0-linux64.tar.gz
 4ccb56fb3700005c9f9188f84152f21a  ./bin/drivers/geckodriver-v0.18.0-linux64.tar.gz
 07cd383c8aef8ea5ef194a506141afd6  ./bin/drivers/geckodriver-v0.19.0-linux32.tar.gz
-785c28da8b681a7e23964f99118b5aab  ./lib/tamper_scripts/obfuscateordinal_encode.py
+145e4a7dc985e99962dabe3b221fc51e  ./lib/tamper_scripts/obfuscateordinal_encode.py
 10bf1bc4ef0287d31633148fab557e8a  ./lib/tamper_scripts/uppercase_encode.py
-5b68de0ce3a783b870921b09b5222146  ./lib/tamper_scripts/hex_encode.py
+fcef22874b6732fd1b1bd062e18e65db  ./lib/tamper_scripts/hex_encode.py
 1537b3b94566aebf0f89bed074e96581  ./lib/tamper_scripts/url_encode.py
 d41d8cd98f00b204e9800998ecf8427e  ./lib/tamper_scripts/__init__.py
 7b636a332b2e99547ec9565d8e094308  ./lib/tamper_scripts/unicode_encode.py
 07a792bccd13f64873a27aee4aaa8ea6  ./lib/tamper_scripts/space2comment_encode.py
 1053a0c89e514d2c94bc822d34715896  ./lib/tamper_scripts/randomcase_encode.py
 349c30cbab4308bd94829d92b4e34f9d  ./lib/tamper_scripts/lowercase_encode.py
-6e1d6276a295f6c5d41b6f6f0e1316b0  ./lib/tamper_scripts/enclosebrackets_encode.py
+0c5e78674a8d27e7c20af1dca8656789  ./lib/tamper_scripts/enclosebrackets_encode.py
-694231b4c2f99406481c34ced85ddfe1  ./lib/tamper_scripts/base64_encode.py
+5824916df46428a8304ee0156bcee989  ./lib/tamper_scripts/multispace2comment_encode.py
-6ac38bec8c32eab57efa01f7a06dff14  ./lib/tamper_scripts/space2null_encode.py
+9fd42d65993aa20d1bf5acbc4d042d2e  ./lib/tamper_scripts/base64_encode.py
+f77b7a9a19b94e26903eeecf5a787ea3  ./lib/tamper_scripts/space2null_encode.py
 3b8c95a6a3b7cecce5118f2fb1ccc6b8  ./lib/tamper_scripts/appendnull_encode.py
+8e8792e38649f18d90bb0084202bb59e  ./lib/tamper_scripts/obfuscateentity_encode.py
 d41d8cd98f00b204e9800998ecf8427e  ./lib/__init__.py
 6299b188a730844954044887f528435a  ./lib/firewall/cloudfront.py
 d41d8cd98f00b204e9800998ecf8427e  ./lib/firewall/__init__.py
-eb3a3066efbcf87dbc10a49be445cb8f  ./lib/firewall/urlscan.py
+81a29a14d72980a306fbaec0dc772048  ./lib/firewall/fortigate.py
 d4986f3d95a773d7c3585b07bcd6310e  ./lib/firewall/sucuri.py
+763af6773e920d6bdc185f5bd4df6084  ./lib/firewall/dw.py
 e4514021485dbb94c3f0023b04af01ad  ./lib/firewall/aws.py
+eb3a3066efbcf87dbc10a49be445cb8f  ./lib/firewall/urlscan.py
+71744d7a95f42063a8fb6e720932cd3d  ./lib/firewall/sonicwall.py
+7af3ee8615c7dc761f050e0ba638eaef  ./lib/firewall/armor.py
 1f303641d59686d544f2986ff74c6b31  ./lib/firewall/webseal.py
+78e6b01feb9bad68c2fc8a79e75930fd  ./lib/firewall/yundun.py
 e4eef006dd909c222b1b9f48826c3ef5  ./lib/firewall/pk.py
+bf5285dc059c761e1719bc734ae8504f  ./lib/firewall/varnish.py
 6b370050b40d8c1d2221424f756c7842  ./lib/firewall/paloalto.py
+73c1727e604ec6e00541687bfc64c0d6  ./lib/firewall/akamai.py
 6bbe2f6f6a2a1ddf0e416e94ec1f0763  ./lib/firewall/siteguard.py
 787e21ed577ff05b095aa0f0e5e5e9bf  ./lib/firewall/cloudflare.py
 c3f01fc8ff7dfe7759f63bf16b00f127  ./lib/firewall/wordfence.py
-6ccf3a1df5aa6429cd3365b7b8ded8f4  ./lib/firewall/powerful.py
+2f0a935d2bb9b8aa711e511f48595a81  ./lib/firewall/powerful.py
-54815706261c32b57fbbdc99244b5cdd  ./lib/firewall/modsecurity.py
-5ee20e2c158d0734b4dd5a8eb47f8ea5  ./lib/firewall/squid.py
-71744d7a95f42063a8fb6e720932cd3d  ./lib/firewall/sonicwall.py
-9070b43428bd17fd5faf86995cb559a2  ./lib/firewall/stringray.py
-bf5285dc059c761e1719bc734ae8504f  ./lib/firewall/varnish.py
-78e6b01feb9bad68c2fc8a79e75930fd  ./lib/firewall/yundun.py
-cb45428e92485b759ff5cb46a0be9c73  ./lib/firewall/yunsuo.py
 bbd8b4c6100070d420d48dc7dfc297eb  ./lib/firewall/webknight.py
+54815706261c32b57fbbdc99244b5cdd  ./lib/firewall/modsecurity.py
+9070b43428bd17fd5faf86995cb559a2  ./lib/firewall/stringray.py
+5ee20e2c158d0734b4dd5a8eb47f8ea5  ./lib/firewall/squid.py
 95b908a21c0ff456ae59df4c6c189c54  ./lib/firewall/wallarm.py
+cb45428e92485b759ff5cb46a0be9c73  ./lib/firewall/yunsuo.py
 8fc8d62377bebbfa7ca4d70a79eab115  ./lib/firewall/bigip.py
 6ea65a0160c21e144e92334acc2e3667  ./lib/firewall/anquanbao.py
-7f4e3ba2f459926fc77bcddc17b933aa  ./lib/firewall/generic.py
+22a0ad8f2fa1a16b651cb5ae37ca9b0d  ./lib/firewall/generic.py
-cf236a16c7869282f55dd4c5ad6347a5  ./lib/attacks/gist_lookup/__init__.py
+ed18ed1f72f3887e63fa7ce060841e4a  ./lib/plugins/aardvark.py
-7183dbd7106ecb436176cebcca4e499f  ./lib/attacks/clickjacking_scan/__init__.py
+a8b3e6924bab72607b1d1c1a8dcb561d  ./lib/plugins/4d.py
+03355a122c047dc598fc271620119978  ./lib/plugins/jquery.py
+d41d8cd98f00b204e9800998ecf8427e  ./lib/plugins/__init__.py
+353db8b22c031433ea73a12943927557  ./lib/plugins/clipbucket.py
+5908a81cc9b332348b26a3ccd5ddb798  ./lib/plugins/ihtml.py
+d76d2839ed8875739328bb5f2a838ba6  ./lib/plugins/360.py
+16e4708c510811760129f6fb4842e92e  ./lib/plugins/3dcart.py
+2ce0a2101bb5706a136de83a729965f3  ./lib/plugins/b2evolution.py
+497facc7b12e6e691aab65980d8f5026  ./lib/plugins/bmcremedy.py
+2dcee5bc924732dd21f16983eef9a99d  ./lib/plugins/abyss.py
+d2c100e6e6f7fbda8448d36a6928c979  ./lib/plugins/68classified.py
+f1eb201cce16853049a615805b01bc60  ./lib/plugins/bomgar.py
+ce3b79dc80e369ffd55d2cbe90e6a0ab  ./lib/plugins/mssqlreportmanager.py
+55ec8cde9d438a90327911910164abf2  ./lib/plugins/opengraph.py
+8658f8a185499ec6d10b1d2da6104c27  ./lib/plugins/atomfeed.py
+c2533d4a8dc5fdaa4b8d584588b32ec2  ./lib/plugins/html5.py
+a3ed012f11ff2bffbc143fbef63d0c12  ./lib/plugins/3com.py
+55d834ae87e96787807e21b65ec68bca  ./lib/plugins/moodle.py
+44019a327ec1db91851d652630788742  ./lib/plugins/googleapi.py
+c4ac50a3f3550c62219e7e4f38d4b496  ./lib/plugins/1024.py
+0b63885649f369ea410c8169e947fdab  ./lib/plugins/accellion.py
+76a1d1decfb872bfafdf510c656f113a  ./lib/plugins/rssfeed.py
+320f0db977c85b477ba1ea78b140cb8a  ./lib/plugins/4images.py
+35dc8b7da4becb60662aab3c48a9210b  ./lib/plugins/openxchange.py
+bdb7ff546787d38bbbd0aac9d4a4cdf8  ./lib/attacks/clickjacking_scan/__init__.py
 d41d8cd98f00b204e9800998ecf8427e  ./lib/attacks/__init__.py
-4c644b0e3a62b6c1528d34a04837aa35  ./lib/attacks/sqlmap_scan/__init__.py
+6e9e0a9e2c72e00d8690c0177b695d56  ./lib/attacks/sqlmap_scan/__init__.py
 5e5bb575014ebe613db6bf671d008cf8  ./lib/attacks/sqlmap_scan/sqlmap_opts.py
-7bc7a6f3e85651aab3588f087563eded  ./lib/attacks/whois_lookup/__init__.py
+d41d8cd98f00b204e9800998ecf8427e  ./lib/attacks/whois_lookup/__init__.py
-d2846e039fefee741db24dd64f7bd50e  ./lib/attacks/whois_lookup/whois.py
+c5b69617f040fef1d5930948905aa8d0  ./lib/attacks/whois_lookup/whois.py
-d2846e039fefee741db24dd64f7bd50e  ./lib/attacks/admin_panel_finder/__init__.py
+4fd96bb3002e949687d7ae863ee87264  ./lib/attacks/admin_panel_finder/__init__.py
-b5cd5e913cc62112776153bdf0f60fa4  ./lib/attacks/xss_scan/__init__.py
+2017e69c3420c9e240fccb310f086da7  ./lib/attacks/xss_scan/__init__.py
-63c45495ec1ed2e98946bef514d8805e  ./lib/attacks/nmap_scan/__init__.py
+40ba04fb18dcbb81cb42376a825c238f  ./lib/attacks/nmap_scan/__init__.py
 216999fa0e84866d5c1d96d5676034e4  ./lib/attacks/nmap_scan/nmap_opts.py
-b6f5f8e43c1e480329b66e193bd91751  ./lib/header_check/__init__.py
+0114ebe3d45612ef143f2777f027374c  ./lib/header_check/__init__.py
-9a2bb0d52f64e12d5a63ce83874ea74a  ./lib/core/common.py
+2a8acb2191d80da75f0e4d09c00df9f6  ./lib/core/common.py
-1faa2b5dfad6eb538bbfe42942d2a9da  ./lib/core/errors.py
+de4254c5e40f7aa4fb81e0608f758a2c  ./lib/core/decorators.py
+3f045c64ef155a517b7a3f3b66905325  ./lib/core/errors.py
 d41d8cd98f00b204e9800998ecf8427e  ./lib/core/__init__.py
-4a87e14ed7a070ae15b1ed7ac7ceaecc  ./lib/core/settings.py
+0faeed8eac30526f3751dd67fe5c9f7e  ./lib/core/settings.py
-4b507b34677b414b8338475fea2c012a  ./lib/core/cache.py
+27bce5d5d1e7d01788c5273016b19370  ./lib/core/parse.py
-9a02e5b913d210350545ac26510a63c9  ./var/search/__init__.py
-8402f23a2586b6f684fb1c3c04c4386f  ./var/search/selenium_search.py
-63ba132381a0cc2d7629852bd5e4aa17  ./var/search/pgp_search.py
 d41d8cd98f00b204e9800998ecf8427e  ./var/__init__.py
 d41d8cd98f00b204e9800998ecf8427e  ./var/auto_issue/__init__.py
-0c11c16126baf789388a661bbbefb149  ./var/auto_issue/github.py
+c58e73857e42a07fa6eb559433b32c1a  ./var/auto_issue/github.py
-7608a7a8195d6d66ebae7664f25639fd  ./var/blackwidow/__init__.py
+503e44b36f0bcd81e20840be5b73320e  ./var/search/__init__.py
+c52867e57beeeeac2da57f597b644faf  ./var/search/selenium_search.py
+12340de27a75273cd444f7257d354311  ./var/search/pgp_search.py
+0af5ab455a535a2f141cfae4758a4bb4  ./var/blackwidow/__init__.py

etc/scripts/install_xvfb.sh

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+
+
+sudo apt-get install xvfb --yes > /dev/null 2>&1

etc/scripts/reinstall.sh

@@ -2,6 +2,7 @@
 
 for pid in $(ps -ef | grep "firefox" | awk '{print $2}'); do kill -9 ${pid}; done > /dev/null 2>&1
 mv ~/.mozilla ~/.mozilla.old > /dev/null 2>&1
+rm /usr/lib/firefox* > /dev/null 2>&1
 sudo apt-get update > /dev/null 2>&1
 sudo apt-get --purge --reinstall --assume-yes install firefox=56.0 > /dev/null 2>&1
-sudo pip2 install selenium -U
+sudo pip2 install selenium -U > /dev/null 2>&1

etc/xml/headers.xml

@@ -5,4 +5,6 @@
     <header name="X-Content-Type-Options"/>
     <header name="Content-Security-Policy"/>
     <header name="Public-Key-Pins"/>
+    <header name="X-Csrf-Token"/>
+    <header name="X-Xsrf-Token"/>
 </headers>

lib/attacks/admin_panel_finder/__init__.py

@@ -7,7 +7,11 @@ try:                 # Python 2
 except ImportError:  # Python 3
     from urllib2 import urlopen, HTTPError
 
-import requests
+from requests.exceptions import (
+    ConnectionError,
+    TooManyRedirects,
+    ReadTimeout
+)
 
 import lib.core.common
 import lib.core.settings
@@ -31,16 +35,22 @@ def check_for_externals(url, data_sep="-" * 30, **kwargs):
     currently_searching = ext[robots if robots else sitemap]
     if verbose:
         lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "currently searching for a '{}'...".format(currently_searching), level=10
+            "currently searching for a '{}'".format(currently_searching), level=10
         ))
+
+    try:
         url = lib.core.settings.replace_http(url)
         full_url = "{}{}{}".format("http://", url, currently_searching)
-    conn = requests.get(full_url)
+        _, code, data, _ = lib.core.common.get_page(full_url)
-    data = conn.content
+    except (TooManyRedirects, ConnectionError, ReadTimeout):
-    code = conn.status_code
+        lib.core.settings.logger.error(lib.core.settings.set_color(
+            "connection to '{}' failed, assuming does not exist and continuing".format(full_url), level=40
+        ))
+        return False
+
     if code == 404:
         lib.core.settings.logger.error(lib.core.settings.set_color(
-            "unable to connect to '{}', assuming does not exist and continuing...".format(
+            "unable to connect to '{}', assuming does not exist and continuing".format(
                 full_url
             ), level=40
         ))
@@ -70,7 +80,7 @@ def check_for_externals(url, data_sep="-" * 30, **kwargs):
                     )
                 )
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "robots.txt page will be saved into a file...", level=25
+            "robots.txt page will be saved into a file", level=25
         ))
         return lib.core.common.write_to_log_file(
             data, lib.core.settings.ROBOTS_PAGE_PATH, lib.core.settings.ROBOTS_TXT_FILENAME.format(
@@ -79,7 +89,7 @@ def check_for_externals(url, data_sep="-" * 30, **kwargs):
         )
     elif sitemap:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "found a sitemap, saving to file...", level=25
+            "found a sitemap, saving to file", level=25
         ))
         return lib.core.common.write_to_log_file(
             data, lib.core.settings.SITEMAP_FILE_LOG_PATH, lib.core.settings.SITEMAP_FILENAME.format(
@@ -104,12 +114,12 @@ def check_for_admin_page(url, exts, protocol="http://", **kwargs):
         true_url = "{}{}{}".format(protocol, stripped_url, ext)
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "trying '{}'...".format(true_url), level=10
+                "trying '{}'".format(true_url), level=10
             ))
         try:
             urlopen(true_url, timeout=5)
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "connected successfully to '{}'...".format(true_url), level=25
+                "connected successfully to '{}'".format(true_url), level=25
             ))
             connections.add(true_url)
         except HTTPError as e:
@@ -117,49 +127,51 @@ def check_for_admin_page(url, exts, protocol="http://", **kwargs):
             if verbose:
                 if "Access Denied" in str(e):
                     lib.core.settings.logger.warning(lib.core.settings.set_color(
-                        "got access denied, possible control panel found without external access on '{}'...".format(
+                        "got access denied, possible control panel found without external access on '{}'".format(
                             true_url
                         ),
                         level=30
                     ))
                     possible_connections.add(true_url)
                 else:
+                    for error_code in lib.core.common.STATUS_CODES.iterkeys():
+                        if int(data[2].split(":")[0]) == error_code:
                             lib.core.settings.logger.error(lib.core.settings.set_color(
-                        "failed to connect got error code {}...".format(
+                                "failed to connect got error code {} (reason: {})".format(
-                            data[2]
+                                     data[2], lib.core.common.STATUS_CODES[error_code]
                                 ), level=40
                              ))
         except Exception as e:
             if verbose:
                 if "<urlopen error timed out>" or "timeout: timed out" in str(e):
                     lib.core.settings.logger.warning(lib.core.settings.set_color(
-                        "connection timed out assuming won't connect and skipping...", level=30
+                        "connection timed out assuming won't connect and skipping", level=30
                     ))
                 else:
                     lib.core.settings.logger.exception(lib.core.settings.set_color(
-                        "failed to connect with unexpected error '{}'...".format(str(e)), level=50
+                        "failed to connect with unexpected error '{}'".format(str(e)), level=50
                     ))
                     request_issue_creation()
     possible_connections, connections = list(possible_connections), list(connections)
-    data_msg = "found {} possible connections(s) and {} successful connection(s)..."
+    data_msg = "found {} possible connections(s) and {} successful connection(s)"
     lib.core.settings.logger.info(lib.core.settings.set_color(
         data_msg.format(len(possible_connections), len(connections))
     ))
     if len(connections) > 0:
         # create the connection tree if we got some connections
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "creating connection tree..."
+            "creating connection tree"
         ))
         lib.core.settings.create_tree(url, connections)
     else:
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
             "did not receive any successful connections to the admin page of "
-            "{}...".format(url), level=50
+            "{}".format(url), level=50
         ))
     if show_possibles:
         if len(possible_connections) > 0:
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "creating possible connection tree..."
+                "creating possible connection tree"
             ))
             lib.core.settings.create_tree(url, possible_connections)
         else:
@@ -169,10 +181,12 @@ def check_for_admin_page(url, exts, protocol="http://", **kwargs):
             ))
     if len(connections) > 0:
         lib.core.settings.logger.warning(lib.core.settings.set_color(
-            "only writing successful connections to log file...", level=30
+            "only writing successful connections to log file", level=30
         ))
         lib.core.common.write_to_log_file(
-            list(connections), lib.core.settings.ADMIN_PAGE_FILE_PATH, lib.core.settings.ADMIN_PAGE_FILE_PATH.format(
+            list(connections),
+            lib.core.settings.ADMIN_PAGE_FILE_PATH,
+            lib.core.settings.ADMIN_PAGE_FILENAME.format(
                 lib.core.settings.replace_http(url)
             )
         )
@@ -197,32 +211,32 @@ def main(url, show=False, verbose=False, **kwargs):
 
     try:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "parsing robots.txt..."
+            "parsing robots.txt"
         ))
         results = check_for_externals(url, robots=True, batch=batch)
         if not results:
             lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "seems like this page is either blocking access to robots.txt or it does not exist...", level=30
+                "seems like this page is either blocking access to robots.txt or it does not exist", level=30
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "checking for a sitemap..."
+            "checking for a sitemap"
         ))
         check_for_externals(url, sitemap=True)
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "loading extensions..."
+            "loading extensions"
         ))
         extensions = __load_extensions()
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "loaded a total of {} extensions...".format(len(extensions)), level=10
+                "loaded a total of {} extensions".format(len(extensions)), level=10
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "attempting to bruteforce admin panel..."
+            "attempting to bruteforce admin panel"
         ))
         if do_threading:
             lib.core.settings.logger.warning(lib.core.settings.set_color(
                 "starting {} threads, you will not be able to end the process until "
-                "it is completed...".format(proc_num), level=30
+                "it is completed".format(proc_num), level=30
             ))
             tasks = []
             for _ in range(0, proc_num):

lib/attacks/clickjacking_scan/__init__.py

@@ -1,5 +1,3 @@
-import requests
-
 import lib.core.common
 import lib.core.settings
 import var.auto_issue.github
@@ -41,7 +39,7 @@ class ClickJackingScanner(object):
                 lib.core.common.HTTP_HEADER.USER_AGENT: agent,
                 lib.core.common.HTTP_HEADER.CONNECTION: "close"
             }
-        req = requests.get(self.url, headers=headers, proxies=lib.core.settings.proxy_string_to_dict(proxy))
+        req, _, _, headers = lib.core.common.get_page(self.url, headers=headers, proxy=proxy)
         headers = req.headers
         if self.safe in headers:
             return False
@@ -67,7 +65,7 @@ def clickjacking_main(url, **kwargs):
             )
             if question.lower().startswith("n"):
                 lib.core.settings.logger.info(lib.core.settings.set_color(
-                    "automatically removing all queries from URL..."
+                    "automatically removing all queries from URL"
                 ))
                 url = "http://{}".format(lib.core.settings.replace_http(url, complete=True))
 
@@ -75,14 +73,14 @@ def clickjacking_main(url, **kwargs):
 
     if verbose:
         lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "generating HTML...", level=10
+            "generating HTML", level=10
         ))
 
     data = scanner.generate_html()
 
     if verbose:
         lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "HTML generated successfully...", level=10
+            "HTML generated successfully", level=10
         ))
         print("{}\n{}\n{}".format("-" * 30, data, "-" * 30))
 
@@ -92,7 +90,7 @@ def clickjacking_main(url, **kwargs):
         if results:
             lib.core.settings.logger.info(lib.core.settings.set_color(
                 "it appears that provided URL '{}' is vulnerable to clickjacking, writing "
-                "to HTML file...".format(url), level=25
+                "to HTML file".format(url), level=25
             ))
             lib.core.common.write_to_log_file(
                 data,
@@ -101,7 +99,7 @@ def clickjacking_main(url, **kwargs):
             )
         else:
             lib.core.settings.logger.error(lib.core.settings.set_color(
-                "provided URL '{}' seems to have the correct protection from clickjacking...".format(
+                "provided URL '{}' seems to have the correct protection from clickjacking".format(
                     url
                 ), level=40
             ))
@@ -111,6 +109,6 @@ def clickjacking_main(url, **kwargs):
     except Exception as e:  # until I figure out the errors, we'll just make issues about them
         lib.core.settings.logger.exception(lib.core.settings.set_color(
             "Zeus failed to process the clickjacking test and received "
-            "error code '{}'...".format(e), level=50
+            "error code '{}'".format(e), level=50
         ))
         var.auto_issue.github.request_issue_creation()

lib/attacks/gist_lookup/__init__.py

@@ -1,159 +0,0 @@
-import re
-import json
-import time
-import requests
-
-import lib.core.settings
-import lib.core.common
-
-
-def __check_remaining_rate_limit():
-    """
-    check how many requests you have left to run
-    """
-    url = lib.core.settings.GITHUB_GIST_SEARCH_URLS["check_rate"]
-    data = requests.get(url, params={lib.core.common.HTTP_HEADER.AUTHORIZATION: "token {}".format(
-        lib.core.settings.get_token(lib.core.settings.GITHUB_AUTH_PATH)
-    )})
-    remaining = data.headers["X-RateLimit-Remaining"]
-    if int(remaining) == 0:
-        lib.core.settings.logger.error(lib.core.settings.set_color(
-            "Github only allows 60 unauthenticated requests per hour, you have hit that limit "
-            "if you need to do more requests it is recommended to run behind a proxy with a different "
-            "user-agent (IE --proxy socks5://127.0.0.1:9050 --random-agent)...", level=40
-        ))
-        lib.core.common.shutdown()
-    else:
-        lib.core.settings.logger.warning(lib.core.settings.set_color(
-            "you have {} unauthenticated requests remaining...".format(remaining), level=30
-        ))
-
-
-def get_raw_data(page_set, proxy=None, agent=None, verbose=False):
-    """
-    parse 10 pages of Github gists and use them
-    """
-    retval = set()
-    url = lib.core.settings.GITHUB_GIST_SEARCH_URLS["search"]
-    headers = {
-        lib.core.common.HTTP_HEADER.USER_AGENT: agent,
-        lib.core.common.HTTP_HEADER.AUTHORIZATION: "token {}".format(lib.core.settings.get_token(lib.core.settings.GITHUB_AUTH_PATH)),
-    }
-    lib.core.settings.logger.info(lib.core.settings.set_color(
-        "searching a total of {} pages of Gists...".format(page_set[-1])
-    ))
-    if proxy is not None:
-        proxy = lib.core.settings.proxy_string_to_dict(proxy)
-    for page in list(page_set):
-        data = requests.get(url.format(page), params=headers, proxies=proxy)
-        # load the found info into JSON format
-        # so we can pull using keys
-        data = json.loads(data.content)
-        for item in data:
-            # get the URL to the raw data so we can search it
-            gist_file = item["files"]
-            gist_filename = gist_file.keys()
-            try:
-                skip_schema = ("-", " ", "")
-                if not any(s == gist_filename for s in skip_schema):
-                    if verbose:
-                        lib.core.settings.logger.debug(lib.core.settings.set_color(
-                            "found filename '{}'...".format(''.join(gist_filename)), level=10
-                        ))
-                    retval.add(gist_file[''.join(gist_filename)]["raw_url"])
-            # sometimes the URL doesn't like being pulled, so we'll just skip those ones
-            except Exception:
-                pass
-    return retval
-
-
-def check_files_for_information(found_url, data_to_search):
-    """
-    check the files to see if they contain any of the information that was specified
-    """
-    # create multiple regex types to ensure that we cover all our
-    # bases while we do the searching.
-    # this will make it so that if there is a match anywhere
-    # in anything, we'll find it.
-    if "www" in data_to_search:
-        # we need to create the query to be just the domain name
-        data_to_search = data_to_search.split(".")[1]
-    data_regex_schema = (
-        re.compile(r"{}".format(data_to_search), re.I),  # a regular match
-        re.compile(r"(http(s)?)(.//)?{}".format(data_to_search), re.I),  # match a URL containing our string
-        re.compile(r"(.)?{}(.)?".format(data_to_search), re.I),  # if it has anything around it
-        re.compile(r"\b{}".format(data_to_search), re.I),  # single boundary match
-        re.compile(r"\b{}\b".format(data_to_search), re.I),  # double boundary match
-        re.compile(r"{}*".format(data_to_search), re.I)  # wildcard match
-    )
-    total_found = set()
-    try:
-        data = requests.get(found_url)
-    except requests.exceptions.ConnectionError:
-        lib.core.settings.logger.warning(lib.core.settings.set_color(
-            "to many requests are being sent to quickly, adding sleep time...", level=30
-        ))
-        time.sleep(3)
-        data = requests.get(found_url)
-    for data_regex in data_regex_schema:
-        if data_regex.search(data.content) is not None:
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "found a match with given specifics, saving full Gist to log file...", level=25
-            ))
-            total_found.add(found_url)
-            lib.core.common.write_to_log_file(
-                data.content, lib.core.settings.GIST_MATCH_LOG, lib.core.settings.GIST_FILENAME
-            )
-    return len(total_found)
-
-
-def github_gist_search_main(query, **kwargs):
-    proxy = kwargs.get("proxy", None)
-    agent = kwargs.get("agent", None)
-    verbose = kwargs.get("verbose", False)
-    thread = kwargs.get("do_threading", False)
-    proc_num = kwargs.get("proc_num", 5)  # TODO:/
-    page_set = kwargs.get("page_set", (1, 2, 3, 4, 5))
-    total_found = 0
-
-    try:
-        if verbose:
-            lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "checking if you have exceeded your search limit...", level=10
-            ))
-        __check_remaining_rate_limit()
-        lib.core.settings.logger.info(lib.core.settings.set_color(
-            "searching Github Gists for '{}'...".format(query)
-        ))
-        gathered_links = get_raw_data(page_set, proxy=proxy, agent=agent, verbose=verbose)
-        lib.core.settings.logger.info(lib.core.settings.set_color(
-            "pulled a total of {} URL's to search...".format(len(gathered_links)), level=25
-        ))
-        if not thread:
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "performing Github Gist search, this will probably take awhile..."
-            ))
-            for url in gathered_links:
-                total = check_files_for_information(url, query)
-                total_found += total
-        else:
-            lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "multi-threading is not implemented yet...", level=35
-            ))
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "performing Github Gist search, this will probably take awhile..."
-            ))
-            for url in gathered_links:
-                total = check_files_for_information(url, query)
-                total_found += total
-        if total_found > 0:
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "found a total of {} interesting Gists...".format(total_found)
-            ))
-        else:
-            lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "did not find any interesting Gists...", level=30
-            ))
-    except KeyboardInterrupt:
-        if not lib.core.common.pause():
-            lib.core.common.shutdown()

lib/attacks/nmap_scan/__init__.py

@@ -1,12 +1,12 @@
 import json
 import socket
-import shlex
-import subprocess
 
 import nmap
 
 import lib.core.common
+import lib.core.errors
 import lib.core.settings
+import lib.core.decorators
 from var.auto_issue.github import request_issue_creation
 
 
@@ -41,11 +41,10 @@ class NmapHook(object):
         """
         send all the information to a JSON file for further use
         """
-        lib.core.settings.create_dir(self.dir)
+        return lib.core.common.write_to_log_file(
-        full_nmap_path = "{}/{}".format(self.dir, self.file.format(self.ip))
+            data, lib.core.settings.NMAP_LOG_FILE_PATH,
-        with open(full_nmap_path, "a+") as log:
+            lib.core.settings.NMAP_FILENAME.format(self.ip)
-            log.write(data)
+        )
-        return full_nmap_path
 
     def show_open_ports(self, json_data, sep="-" * 30):
         """
@@ -53,12 +52,14 @@ class NmapHook(object):
         """
         # have to create a spacer or the output comes out funky..
         spacer_data = {4: " " * 8, 6: " " * 6, 8: " " * 4}
-        lib.core.settings.logger.info(lib.core.settings.set_color("finding data for IP '{}'...".format(self.ip)))
+        lib.core.settings.logger.info(lib.core.settings.set_color("finding data for IP '{}'".format(self.ip)))
         json_data = json.loads(json_data)["scan"]
+        host = json_data[self.ip]["hostnames"][0]["name"]
+        host_skip = (not len(host) == 0, " ", "", None)
         print(
             "{}\nScanned: {} ({})\tStatus: {}\nProtocol: {}\n".format(
                 sep, self.ip,
-                json_data[self.ip]["hostnames"][0]["name"],
+                host if host != any(s for s in list(host_skip)) else "unknown",
                 json_data[self.ip]["status"]["state"],
                 "TCP"
             )
@@ -91,27 +92,42 @@ def perform_port_scan(url, scanner=NmapHook, **kwargs):
     """
     verbose = kwargs.get("verbose", False)
     opts = kwargs.get("opts", None)
+    timeout_time = kwargs.get("timeout", None)
 
+    if timeout_time is None:
+        timeout_time = 120
+
+    with lib.core.decorators.TimeOut(seconds=timeout_time):
+        lib.core.settings.logger.warning(lib.core.settings.set_color(
+            "if the port scan is not completed in {}(m) it will timeout".format(
+                lib.core.settings.convert_to_minutes(timeout_time)
+            ), level=30
+        ))
         url = url.strip()
         lib.core.settings.logger.info(lib.core.settings.set_color(
-        "attempting to find IP address for hostname '{}'...".format(url)
+            "attempting to find IP address for hostname '{}'".format(url)
         ))
+
+        try:
             found_ip_address = socket.gethostbyname(url)
-    lib.core.settings.logger.info(lib.core.settings.set_color(
+        except socket.gaierror:
-        "found IP address for given URL -> '{}'...".format(found_ip_address), level=25
+            lib.core.settings.logger.fatal(lib.core.settings.set_color(
+                "failed to gather IP address for URL '{}'".format(url)
             ))
+            return
+
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "checking for nmap on your system...", level=10
+                "checking for nmap on your system", level=10
             ))
         nmap_exists = "".join(find_nmap())
         if nmap_exists:
             if verbose:
                 lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "nmap has been found under '{}'...".format(nmap_exists), level=10
+                    "nmap has been found under '{}'".format(nmap_exists), level=10
                 ))
             lib.core.settings.logger.info(lib.core.settings.set_color(
-            "starting port scan on IP address '{}'...".format(found_ip_address)
+                "starting port scan on IP address '{}'".format(found_ip_address)
             ))
             try:
                 data = scanner(found_ip_address, opts=opts)
@@ -119,38 +135,35 @@ def perform_port_scan(url, scanner=NmapHook, **kwargs):
                 data.show_open_ports(json_data)
                 file_path = data.send_to_file(json_data)
                 lib.core.settings.logger.info(lib.core.settings.set_color(
-                "port scan completed, all data saved to JSON file under '{}'...".format(file_path)
+                    "port scan completed, all data saved to JSON file under '{}'".format(file_path)
                 ))
             except KeyError:
                 lib.core.settings.logger.fatal(lib.core.settings.set_color(
-                "no port information found for '{}({})'...".format(
+                    "no port information found for '{}({})'".format(
                         url, found_ip_address
                     ), level=50
                 ))
             except KeyboardInterrupt:
                 if not lib.core.common.pause():
                     lib.core.common.shutdown()
+            except lib.core.errors.PortScanTimeOutException:
+                lib.core.settings.logger.error(lib.core.settings.set_color(
+                    "port scan is taking to long and has hit the timeout, you "
+                    "can increase this time by passing the --time-sec flag (IE "
+                    "--time-sec 300)", level=40
+                ))
             except Exception as e:
                 lib.core.settings.logger.exception(lib.core.settings.set_color(
-                "ran into exception '{}', cannot continue quitting...".format(e), level=50
+                    "ran into exception '{}', cannot continue quitting".format(e), level=50
                 ))
                 request_issue_creation()
                 pass
         else:
             lib.core.settings.logger.fatal(lib.core.settings.set_color(
-            "nmap was not found on your system...", level=50
+                "nmap was not found on your system", level=50
             ))
-        question = lib.core.common.prompt(
+            lib.core.common.run_fix(
-            "would you like to automatically install it", opts="yN"
+                "would you like to automatically install it",
+                "sudo sh {}".format(lib.core.settings.NMAP_INSTALLER_TOOL),
+                "nmap is not installed, please install it in order to continue"
             )
-        if question.lower().startswith("y"):
-            install_nmap_command = shlex.split("sudo sh {}".format(lib.core.settings.NMAP_INSTALLER_TOOL))
-            subprocess.call(install_nmap_command)
-            lib.core.settings.logger.info(lib.core.settings.set_color(
-                "nmap has been successfully installed, re-running...", level=25
-            ))
-            perform_port_scan(url, verbose=verbose, opts=opts)
-        else:
-            lib.core.settings.logger.fatal(lib.core.settings.set_color(
-                "nmap is not installed, please install it in order to continue...", level=50
-            ))

lib/attacks/sqlmap_scan/__init__.py

@@ -111,7 +111,7 @@ class SqlmapHook(object):
         if current_status != "running":
             raise lib.core.errors.SqlmapFailedStart(
                 "sqlmap API failed to start the run, check the client and see what "
-                "the problem is and try again..."
+                "the problem is and try again"
             )
         already_displayed = set()
         while current_status == "running":
@@ -156,7 +156,7 @@ def sqlmap_scan_main(url, port=None, verbose=None, opts=None, auto_start=False):
 
     if auto_start:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "attempting to launch sqlmap API..."
+            "attempting to launch sqlmap API"
         ))
         sqlmap_api_command = shlex.split("sudo sh {} p {}".format(
             lib.core.settings.LAUNCH_SQLMAP_API_TOOL, "".join(found_path)
@@ -164,58 +164,58 @@ def sqlmap_scan_main(url, port=None, verbose=None, opts=None, auto_start=False):
         subprocess.Popen(sqlmap_api_command, stdout=subprocess.PIPE)
         if is_started:
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "sqlmap API is up and running, continuing process..."
+                "sqlmap API is up and running, continuing process"
             ))
         else:
             lib.core.settings.logger.error(lib.core.settings.set_color(
-                "there was a problem starting sqlmap API...", level=40
+                "there was a problem starting sqlmap API", level=40
             ))
             lib.core.common.prompt(
-                "manually start the API and press enter when ready..."
+                "manually start the API and press enter when ready"
             )
     else:
         if not is_started:
             lib.core.common.prompt(
-                "sqlmap API is not started, start it and press enter to continue..."
+                "sqlmap API is not started, start it and press enter to continue"
             )
     try:
         sqlmap_scan = SqlmapHook(url, port=port)
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "initializing new sqlmap scan with given URL '{}'...".format(url)
+            "initializing new sqlmap scan with given URL '{}'".format(url)
         ))
         sqlmap_scan.init_new_scan()
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "scan initialized...", level=10
+                "scan initialized", level=10
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "gathering sqlmap API scan ID..."
+            "gathering sqlmap API scan ID"
         ))
         api_id = sqlmap_scan.get_scan_id()
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "current sqlmap scan ID: '{}'...".format(api_id), level=10
+                "current sqlmap scan ID: '{}'".format(api_id), level=10
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "starting sqlmap scan on url: '{}'...".format(url), level=25
+            "starting sqlmap scan on url: '{}'".format(url), level=25
         ))
         if opts:
             if verbose:
                 lib.core.settings.logger.debug(lib.core.settings.set_color(
-                    "using arguments: '{}'...".format(opts), level=10
+                    "using arguments: '{}'".format(opts), level=10
                 ))
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "adding arguments to sqlmap API..."
+                "adding arguments to sqlmap API"
             ))
         else:
             if verbose:
                 lib.core.settings.logger.debug(lib.core.settings.set_color(
-                    "no arguments passed, skipping...", level=10
+                    "no arguments passed, skipping", level=10
                 ))
         lib.core.settings.logger.warning(lib.core.settings.set_color(
             "please keep in mind that this is the API, output will "
             "not be saved to log file, it may take a little longer "
-            "to finish processing, launching sqlmap...", level=30
+            "to finish processing, launching sqlmap", level=30
         ))
         sqlmap_scan.start_scan(api_id, opts=opts)
         print("-" * 30)
@@ -224,7 +224,7 @@ def sqlmap_scan_main(url, port=None, verbose=None, opts=None, auto_start=False):
     except requests.exceptions.HTTPError as e:
         lib.core.settings.logger.exception(lib.core.settings.set_color(
             "ran into error '{}', seems you didn't start the server, check "
-            "the server port and try again...".format(e), level=50
+            "the server port and try again".format(e), level=50
         ))
         pass
     except KeyboardInterrupt:
@@ -236,7 +236,7 @@ def sqlmap_scan_main(url, port=None, verbose=None, opts=None, auto_start=False):
                 "sqlmap API is not started, did you forget to start it? "
                 "You will need to open a new terminal, cd into sqlmap, and "
                 "run `python sqlmapapi.py -s` otherwise pass the correct flags "
-                "to auto start the API...", level=40
+                "to auto start the API", level=40
             ))
             pass
         else:

lib/attacks/whois_lookup/whois.py

@@ -74,23 +74,22 @@ def whois_lookup_main(domain, **kwargs):
 
     try:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "performing WhoIs lookup on given domain '{}'...".format(domain)
+            "performing WhoIs lookup on given domain '{}'".format(domain)
         ))
         if timeout is not None:
             time.sleep(timeout)
         try:
             raw_information = gather_raw_whois_info(domain)
-        except Exception as e:
+        except Exception:
-            lib.core.settings.logger.exception(str(e))
             lib.core.settings.logger.error(lib.core.settings.set_color(
-                "unable to produce information from WhoIs lookup...", level=40
+                "unable to produce information from WhoIs lookup", level=40
             ))
             return None
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "discovered raw information...", level=25
+            "discovered raw information", level=25
         ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "gathering interesting information..."
+            "gathering interesting information"
         ))
         interesting_data = get_interesting(raw_information)
         if verbose:
@@ -98,7 +97,7 @@ def whois_lookup_main(domain, **kwargs):
                 human_readable_display(domain, interesting_data)
             except (ValueError, Exception):
                 lib.core.settings.logger.error(lib.core.settings.set_color(
-                    "unable to display any information from WhoIs lookup on domain '{}'...".format(domain), level=50
+                    "unable to display any information from WhoIs lookup on domain '{}'".format(domain), level=50
                 ))
                 return None
         lib.core.common.write_to_log_file(

lib/attacks/xss_scan/__init__.py

@@ -11,6 +11,7 @@ import requests
 
 import lib.core.common
 import lib.core.settings
+import lib.core.decorators
 from lib.core.errors import InvalidTamperProvided
 
 
@@ -28,6 +29,19 @@ def list_tamper_scripts(path="{}/lib/tamper_scripts"):
     return retval
 
 
+def assign_protocol(url, force=False):
+    auto_assign = ("http://{}", "https://{}")
+    url_verification = re.compile(r"http(s)?", re.I)
+
+    if url_verification.search(url) is None:
+        if not force:
+            return auto_assign[0].format(url)
+        else:
+            return auto_assign[1].format(url)
+    else:
+        return url
+
+
 def __tamper_payload(payload, tamper_type, warning=True, **kwargs):
     """
     add the tamper to the payload from the given tamper type
@@ -39,7 +53,7 @@ def __tamper_payload(payload, tamper_type, warning=True, **kwargs):
         if warning:
             if verbose:
                 lib.core.settings.logger.debug(lib.core.settings.set_color(
-                    "tampering payload with '{}'...".format(tamper), level=10
+                    "tampering payload with '{}'".format(tamper), level=10
                 ))
         if tamper in acceptable:
             tamper_name = "lib.tamper_scripts.{}_encode"
@@ -57,7 +71,7 @@ def __load_payloads(filename="{}/etc/text_files/xss_payloads.txt"):
     with open(filename.format(os.getcwd())) as payloads: return payloads.readlines()
 
 
-def create_urls(url, payload_list, tamper=None, verbose=False):
+def create_urls(url, payload_list, tamper=None, verbose=False, force=False):
     """
     create the tampered URL's, write them to a temporary file and read them from there
     """
@@ -72,13 +86,12 @@ def create_urls(url, payload_list, tamper=None, verbose=False):
                     else:
                         payload = __tamper_payload(payload, tamper_type=tamper, warning=False, verbose=verbose)
                 except InvalidTamperProvided:
-                    lib.core.settings.logger.error(lib.core.settings.set_color(
+                    lib.core.settings.logger.warning(lib.core.settings.set_color(
-                        "you provided and invalid tamper script, acceptable tamper scripts are: {}...".format(
+                        "you provided and invalid tamper script, acceptable tamper scripts are: {}".format(
                             " | ".join(list_tamper_scripts()), level=40
                         )
                     ))
-                    lib.core.common.shutdown()
+            loaded_url = "{}{}\n".format(assign_protocol(url.strip(), force=force), payload.strip())
-            loaded_url = "{}{}\n".format(url.strip(), payload.strip())
             tmp.write(loaded_url)
     return tf_name
 
@@ -96,7 +109,7 @@ def find_xss_script(url, **kwargs):
     else:
         retval = data[payload_parser["query"]]
 
-    # just double checking...
+    # just double checking
     if retval == "" or None:
         retval = data[payload_parser["path"]]
     return retval
@@ -108,15 +121,9 @@ def scan_xss(url, agent=None, proxy=None):
     chance that the URL is vulnerable to XSS attacks. Usually what will happen is the payload will
     be tampered or encoded if the site is not vulnerable
     """
-    user_agent = agent or lib.core.settings.DEFAULT_USER_AGENT
+
-    config_proxy = lib.core.settings.proxy_string_to_dict(proxy)
+    try:
-    config_headers = {
+        _, status, html_data, _ = lib.core.common.get_page(url, agent=agent, proxy=proxy)
-        lib.core.common.HTTP_HEADER.CONNECTION: "close",
-        lib.core.common.HTTP_HEADER.USER_AGENT: user_agent
-    }
-    xss_request = requests.get(url, proxies=config_proxy, headers=config_headers)
-    status = xss_request.status_code
-    html_data = xss_request.content
         query = find_xss_script(url)
         for db in lib.core.settings.DBMS_ERRORS.keys():
             for item in lib.core.settings.DBMS_ERRORS[db]:
@@ -126,6 +133,8 @@ def scan_xss(url, agent=None, proxy=None):
             if query in html_data:
                 return True, None
         return False, None
+    except (requests.exceptions.ChunkedEncodingError, requests.exceptions.ConnectionError):
+        return False, None
 
 
 def main_xss(start_url, proxy=None, agent=None, **kwargs):
@@ -135,71 +144,100 @@ def main_xss(start_url, proxy=None, agent=None, **kwargs):
     tamper = kwargs.get("tamper", None)
     verbose = kwargs.get("verbose", False)
     batch = kwargs.get("batch", False)
+    force = kwargs.get("force_ssl", False)
+
+    question_msg = (
+        "it appears that heuristic tests have shown this URL may not be a good "
+        "candidate to perform XSS tests on, would you like to continue anyways"
+    )
+    if not batch:
+        question = lib.core.common.prompt(
+            question_msg, opts="yN"
+        ) if not lib.core.settings.URL_QUERY_REGEX.match(start_url) else "y"
+    else:
+        question = lib.core.common.prompt(
+            question_msg, opts="yN", default="y"
+        )
+
+    if not question.lower().startswith("y"):
+        return
 
     try:
         if tamper:
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "tampering payloads with '{}'...".format(tamper)
+                "tampering payloads with '{}'".format(tamper)
             ))
         find_xss_script(start_url)
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "loading payloads..."
+            "loading payloads"
         ))
         payloads = __load_payloads()
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "a total of {} payloads loaded...".format(len(payloads)), level=10
+                "a total of {} payloads loaded".format(len(payloads)), level=10
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "payloads will be written to a temporary file and read from there..."
+            "payloads will be written to a temporary file and read from there"
         ))
-        filename = create_urls(start_url, payloads, tamper=tamper, verbose=verbose)
+        filename = create_urls(start_url, payloads, tamper=tamper, verbose=verbose, force=force)
         lib.core.settings.logger.info(lib.core.settings.set_color(
-                "loaded URL's have been saved to '{}'...".format(filename), level=25
+                "loaded URL's have been saved to '{}'".format(filename), level=25
             ))
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "testing for XSS vulnerabilities on host '{}'...".format(start_url)
+            "testing for XSS vulnerabilities on host '{}'".format(start_url)
         ))
         if proxy is not None:
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "using proxy '{}'...".format(proxy)
+                "using proxy '{}'".format(proxy)
             ))
         success = set()
         with open(filename) as urls:
             for i, url in enumerate(urls.readlines(), start=1):
                 url = url.strip()
-                result = scan_xss(url, proxy=proxy, agent=agent)
                 payload = find_xss_script(url)
+                try:
+                    result = scan_xss(url, proxy=proxy, agent=agent)
                     if verbose:
                         lib.core.settings.logger.info(lib.core.settings.set_color(
-                        "trying payload '{}'...".format(payload)
+                            "trying payload '{}'".format(payload)
                         ))
                     if result[0] != "sqli" and result[0] is True:
                         success.add(url)
                         if verbose:
                             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                            "payload '{}' appears to be usable...".format(payload), level=15
+                                "payload '{}' appears to be usable".format(payload), level=15
                             ))
                     elif result[0] is "sqli":
                         if i <= 1:
                             lib.core.settings.logger.error(lib.core.settings.set_color(
-                            "loaded URL '{}' threw a DBMS error and appears to be injectable, test for SQL injection, "
+                                "loaded URL '{}' threw a DBMS error and appears to be injectable, test for "
-                            "backend DBMS appears to be '{}'...".format(
+                                "SQL injection, backend DBMS appears to be '{}'".format(
                                     url, result[1]
                                 ), level=40
                             ))
                         else:
                             if verbose:
                                 lib.core.settings.logger.error(lib.core.settings.set_color(
-                                "SQL error discovered...", level=40
+                                    "SQL error discovered", level=40
                                 ))
                     else:
                         if verbose:
                             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                            "host '{}' does not appear to be vulnerable to XSS attacks with payload '{}'...".format(
+                                "host '{}' does not appear to be vulnerable to XSS attacks with payload '{}'".format(
                                     start_url, payload
                                 ), level=10
                             ))
+                except (
+                        requests.exceptions.ConnectionError,
+                        requests.exceptions.TooManyRedirects,
+                        requests.exceptions.ReadTimeout,
+                        requests.exceptions.InvalidURL
+                ):
+                    if not payload == "":
+                        lib.core.settings.logger.error(lib.core.settings.set_color(
+                            "payload '{}' caused a connection error, assuming no good and continuing".format(payload), level=40
+                        ))
+
         if len(success) != 0:
             lib.core.settings.logger.info(lib.core.settings.set_color(
                 "possible XSS scripts to be used:", level=25
@@ -207,9 +245,9 @@ def main_xss(start_url, proxy=None, agent=None, **kwargs):
             lib.core.settings.create_tree(start_url, list(success))
         else:
             lib.core.settings.logger.error(lib.core.settings.set_color(
-                "host '{}' does not appear to be vulnerable to XSS attacks...".format(start_url)
+                "host '{}' does not appear to be vulnerable to XSS attacks".format(start_url), level=40
             ))
-        question_msg = "would you like to keep the URL's saved for further testing"
+        question_msg = "would you like to keep the created URLs saved for further testing"
         if not batch:
             save = lib.core.common.prompt(
                 question_msg, opts="yN"

lib/core/cache.py

@@ -1,27 +0,0 @@
-from functools import wraps
-
-import lib.core.settings
-
-
-def cache(func):
-    """
-    if we come across the same URL more then once, it will be cached into memory
-    so that we don't have to test it again
-    """
-    __cache = {}
-
-    @wraps(func)
-    def func_wrapper(*args, **kwargs):
-        if args in __cache:
-            lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "cached detection has shown that the target URL WAF/IPS/IDS is '{}'...".format(
-                    __cache[args]
-                ), level=35
-            ))
-            return __cache[args]
-        else:
-            __to_cache = func(*args, **kwargs)
-            __cache[args] = __to_cache
-            return __to_cache
-
-    return func_wrapper

lib/core/common.py

@@ -2,6 +2,8 @@ import os
 import re
 import json
 import time
+import shlex
+import subprocess
 try:
     from urllib import (  # python 2
         unquote
@@ -11,17 +13,34 @@ except ImportError:
         unquote
     )
 
+import requests
 from lxml import etree
 
 import lib.core.settings
 
 # reference https://en.wikipedia.org/wiki/List_of_HTTP_status_codes
 STATUS_CODES = {
-    200: "OK", 300: "multiple choices", 302: "not found",
+    100: "continue", 101: "switching protocols", 102: "processing",
-    400: "bad request", 401: "unauthorized", 403: "forbidden",
+    200: "OK", 201: "created", 202: "accepted", 203: "non-authoritative information",
-    404: "not found", 405: "method not allowed", 500: "internal server error",
+    204: "no content", 205: "reset content", 206: "partial content",
-    501: "not implemented", 502: "bad gateway", 503: "service unavailable",
+    207: "multi-status", 208: "already reported", 226: "IM used",
-    "other": "unexpected error code"
+    300: "multiple choices", 301: "moved permanently",  302: "found redirect",
+    303: "see other", 304: "not modified", 305: "use proxy",
+    306: "switch proxy", 308: "permanent redirect",
+    400: "bad request", 401: "unauthorized", 402: "payment required",
+    403: "forbidden", 404: "not found", 405: "method not allowed",
+    406: "not acceptable", 407: "proxy authentication required", 408: "request timed out",
+    409: "conflict", 410: "gone", 411: "length required", 412: "precondition failed",
+    413: "payload to large", 414: "URI too long", 415: "unsupported media type",
+    416: "range not satisfiable", 417: "expectation failed", 418: "im a teapot {EASTER EGG!}",
+    421: "misdirected request", 422: "unprocesseable entity", 423: "locked",
+    424: "failed dependency", 426: "upgrade requried", 428: "precondition required",
+    429: "to many requests", 431: "request headers field too large",
+    451: "unavailable for legal reasons",
+    500: "internal server error", 501: "not implemented", 502: "bad gateway",
+    503: "service unavailable", 504: "gateway timeout", 505: "HTTP version not supported",
+    506: "variant also negotiable", 507: "insufficient storage", 508: "loop detected",
+    510: "not extended", 511: "network authentication required", "other": "unexpected error code"
 }
 
 
@@ -123,7 +142,7 @@ def write_to_log_file(data_to_write, path, filename, blacklist=False):
     )
     skip_log_schema = (
         "url-log", "blackwidow-log", "zeus-log",
-        "extracted", ".blacklist", "gist-match"
+        "extracted", ".blacklist", "sqli-sites"
     )
     to_search = filename.split("-")[0]
     amount = len([f for f in os.listdir(path) if to_search in f])
@@ -143,7 +162,7 @@ def write_to_log_file(data_to_write, path, filename, blacklist=False):
             items = log.readlines()
             if any(d.strip() == data_to_write for d in items):
                 lib.core.settings.logger.info(lib.core.settings.set_color(
-                    "query already in blacklist..."
+                    "query already in blacklist"
                 ))
                 return full_file_path
             else:
@@ -162,7 +181,7 @@ def write_to_log_file(data_to_write, path, filename, blacklist=False):
             else:
                 log.write(data_to_write + "\n")
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "successfully wrote found items to '{}'...".format(full_file_path)
+        "successfully wrote found items to '{}'".format(full_file_path)
     ))
     return full_file_path
 
@@ -243,3 +262,79 @@ def pause():
     return prompt(
         message, paused=True
     )
+
+
+def run_fix(message, command, fail_message, exit_process=False):
+    """
+    run the fix script for the program
+    """
+    do_fix = prompt(
+        message, opts="yN"
+    )
+    if do_fix.lower().startswith("y"):
+        cmd = shlex.split(command)
+        subprocess.call(cmd)
+        if exit_process:
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "command completed successfully, should be safe to re-run Zeus"
+            ))
+    else:
+        lib.core.settings.logger.fatal(lib.core.settings.set_color(
+            fail_message, level=50
+        ))
+
+
+def get_page(url, **kwargs):
+    agent = kwargs.get("agent", None)
+    proxy = kwargs.get("proxy", None)
+    xforward = kwargs.get("xforward", False)
+    auth = kwargs.get("auth", None)
+    skip_verf = kwargs.get("skip_verf", False)
+
+    if agent is None:
+        agent = lib.core.settings.DEFAULT_USER_AGENT
+
+    if xforward:
+        ip_list = (
+            lib.core.settings.create_random_ip(),
+            lib.core.settings.create_random_ip(),
+            lib.core.settings.create_random_ip()
+        )
+        headers = {
+            HTTP_HEADER.CONNECTION: "close",
+            HTTP_HEADER.USER_AGENT: agent,
+            HTTP_HEADER.X_FORWARDED_FOR: "{}, {}, {}".format(
+                ip_list[0], ip_list[1], ip_list[2]
+            )
+        }
+    elif auth:
+        headers = {
+            HTTP_HEADER.CONNECTION: "close",
+            HTTP_HEADER.USER_AGENT: agent,
+            HTTP_HEADER.AUTHORIZATION: "{}".format(
+                auth
+            )
+        }
+    else:
+        headers = {
+            HTTP_HEADER.CONNECTION: "close",
+            HTTP_HEADER.USER_AGENT: agent
+        }
+
+    if proxy is not None:
+        proxies = {
+            "https": proxy,
+            "http": proxy
+        }
+    else:
+        proxies = {}
+
+    if proxy is not None and "127.0.0.1" in proxy:
+        req = requests.get(url, params=headers, proxies=proxies, verify=False, timeout=40)
+    else:
+        req = requests.get(url, params=headers, proxies=proxies, verify=False, timeout=20)
+
+    status = req.status_code
+    html = req.content
+    headers = req.headers
+    return req, status, html, headers

lib/core/decorators.py

@@ -0,0 +1,41 @@
+import signal
+from functools import wraps
+
+import lib.core.errors
+import lib.core.settings
+
+
+class TimeOut:
+
+    def __init__(self, seconds=1, error_message='Timeout'):
+        self.seconds = seconds
+        self.error_message = error_message
+
+    def handle_timeout(self, signum, frame):
+        raise lib.core.errors.PortScanTimeOutException(self.error_message)
+
+    def __enter__(self):
+        signal.signal(signal.SIGALRM, self.handle_timeout)
+        signal.alarm(self.seconds)
+
+    def __exit__(self, type_, value, traceback):
+        signal.alarm(0)
+
+
+def cache(func):
+    """
+    if we come across the same URL more then once, it will be cached into memory
+    so that we don't have to test it again
+    """
+    __cache = {}
+
+    @wraps(func)
+    def func_wrapper(*args, **kwargs):
+        if args in __cache:
+            return __cache[args]
+        else:
+            __to_cache = func(*args, **kwargs)
+            __cache[args] = __to_cache
+            return __to_cache
+
+    return func_wrapper

lib/core/errors.py

@@ -17,3 +17,9 @@ class InvalidInputProvided(Exception): pass
 
 
 class InvalidTamperProvided(Exception): pass
+
+
+class PortScanTimeOutException(Exception): pass
+
+
+class ZeusArgumentException(Exception): pass

lib/core/parse.py

@@ -0,0 +1,295 @@
+import sys
+from optparse import (
+    OptionParser,
+    OptionGroup,
+    SUPPRESS_HELP
+)
+
+import lib.core.settings
+import lib.core.common
+import lib.core.errors
+import lib.attacks.nmap_scan.nmap_opts
+import lib.attacks.sqlmap_scan.sqlmap_opts
+
+
+class ZeusParser(OptionParser):
+
+    """
+    Zeus's option parser
+    """
+
+    def __init__(self):
+        OptionParser.__init__(self)
+
+    @staticmethod
+    def cmd_parser():
+        """
+        command line parser, parses all of Zeus's arguments and flags
+        """
+        parser = OptionParser(usage="./zeus.py -d|r|l|f|b DORK|FILE|URL [ATTACKS] [--OPTS]")
+
+        # mandatory options
+        mandatory = OptionGroup(parser, "Mandatory Options",
+                                "These options have to be used in order for Zeus to run")
+
+        mandatory.add_option("-d", "--dork", dest="dorkToUse", metavar="DORK",
+                             help="Specify a singular Google dork to use for queries")
+
+        mandatory.add_option("-l", "--dork-list", dest="dorkFileToUse", metavar="FILE-PATH",
+                             help="Specify a file full of dorks to run through")
+
+        mandatory.add_option("-r", "--rand-dork", dest="useRandomDork", action="store_true",
+                             help="Use a random dork from the etc/dorks.txt file to perform the scan")
+
+        mandatory.add_option("-b", "--blackwidow", dest="spiderWebSite", metavar="URL",
+                             help="Spider a single webpage for all available URL's")
+
+        mandatory.add_option("-f", "--url-file", dest="fileToEnumerate", metavar="FILE-PATH",
+                             help="Run an attack on URL's in a given file")
+
+        # being worked on
+        # TODO:/
+        mandatory.add_option("-u", "--url", dest="singleTargetRecon", metavar="URL",
+                             help=SUPPRESS_HELP)
+
+        # attack options
+        attacks = OptionGroup(parser, "Attack arguments",
+                              "These arguments will give you the choice on how you want to check the websites")
+
+        attacks.add_option("-s", "--sqli", dest="runSqliScan", action="store_true",
+                           help="Run a Sqlmap SQLi scan on the discovered URL's")
+
+        attacks.add_option("-p", "--port-scan", dest="runPortScan", action="store_true",
+                           help="Run a Nmap port scan on the discovered URL's")
+
+        attacks.add_option("-a", "--admin-panel", dest="adminPanelFinder", action="store_true",
+                           help="Search for the websites admin panel")
+
+        attacks.add_option("-x", "--xss-scan", dest="runXssScan", action="store_true",
+                           help="Run an XSS scan on the found URL's")
+
+        attacks.add_option("-w", "--whois-lookup", dest="performWhoisLookup", action="store_true",
+                           help="Perform a WhoIs lookup on the provided domain")
+
+        attacks.add_option("-c", "--clickjacking", dest="performClickjackingScan", action="store_true",
+                           help="Perform a clickjacking scan on a provided URL")
+
+        # being worked on
+        # TODO:/
+        attacks.add_option("-g", "--github-search", dest="searchGithub", action="store_true",
+                           help=SUPPRESS_HELP)
+
+        attacks.add_option("-P", "--pgp", dest="pgpLookup", action="store_true",
+                           help="Perform a PGP public key lookup on the found URLs")
+
+        attacks.add_option("--sqlmap-args", dest="sqlmapArguments", metavar="SQLMAP-ARGS",
+                           help="Pass the arguments to send to the sqlmap API within quotes & "
+                                "separated by a comma. IE 'dbms mysql, verbose 3, level 5'")
+
+        attacks.add_option("--sqlmap-conf", dest="sqlmapConfigFile", metavar="CONFIG-FILE-PATH",
+                           help="Pass a configuration file that contains the sqlmap arguments")
+
+        attacks.add_option("--nmap-args", dest="nmapArguments", metavar="NMAP-ARGS",
+                           help="Pass the arguments to send to the nmap API within quotes & "
+                                "separated by a pipe. IE '-O|-p 445, 1080'")
+
+        attacks.add_option("--show-sqlmap", dest="showSqlmapArguments", action="store_true",
+                           help="Show the arguments that the sqlmap API understands")
+
+        attacks.add_option("--show-nmap", dest="showNmapArgs", action="store_true",
+                           help="Show the arguments that nmap understands")
+
+        attacks.add_option("--show-possibles", dest="showAllConnections", action="store_true",
+                           help="Show all connections made during the admin panel search")
+
+        attacks.add_option("--tamper", dest="tamperXssPayloads", metavar="TAMPER-SCRIPT",
+                           help="Send the XSS payloads through tampering before sending to the target")
+
+        # being worked on
+        # TODO:/
+        attacks.add_option("--thread", dest="threadPanels", action="store_true",
+                           help=SUPPRESS_HELP)
+
+        attacks.add_option("--auto", dest="autoStartSqlmap", action="store_true",
+                           help="Automatically start the sqlmap API (or at least try to)")
+
+        # search engine options
+        engines = OptionGroup(parser, "Search engine arguments",
+                              "Arguments to change the search engine used (default is Google)")
+
+        engines.add_option("-D", "--search-engine-ddg", dest="useDDG", action="store_true",
+                           help="Use DuckDuckGo as the search engine")
+
+        engines.add_option("-B", "--search-engine-bing", dest="useBing", action="store_true",
+                           help="Use Bing as the search engine")
+
+        engines.add_option("-A", "--search-engine-aol", dest="useAOL", action="store_true",
+                           help="Use AOL as the search engine")
+
+        # arguments to edit your search patterns
+        search_items = OptionGroup(parser, "Search options",
+                                   "Arguments that will control the search criteria")
+
+        search_items.add_option("-L", "--links", dest="amountToSearch", type=int, metavar="HOW-MANY-LINKS",
+                                help="Specify how many links to try and search on Google")
+
+        search_items.add_option("-M", "--multi", dest="searchMultiplePages", action="store_true",
+                                help="Search multiple pages of Google")
+
+        search_items.add_option("-E", "--exclude-none", dest="noExclude", action="store_true",
+                                help="Do not exclude URLs because they do not have a GET(query) parameter in them")
+
+        search_items.add_option("-W", "--webcache", dest="parseWebcache", action="store_true",
+                                help="Parse webcache URLs for the redirect in them")
+
+        search_items.add_option("--x-forward", dest="forwardedForRandomIP", action="store_true",
+                                help="Add a header called 'X-Forwarded-For' with three random IP addresses")
+
+        search_items.add_option("--time-sec", dest="controlTimeout", metavar="SECONDS", type=int,
+                                help="Control the sleep and timeout times in relevant situations")
+
+        search_items.add_option("--identify-waf", dest="identifyProtection", action="store_true",
+                                help="Attempt to identify if the target is protected by some kind of "
+                                     "WAF/IDS/IPS")
+
+        # being worked on
+        # TODO:/
+        search_items.add_option("--force-ssl", dest="forceSSL", action="store_true",
+                                help=SUPPRESS_HELP)
+
+        search_items.add_option("--identify-plugins", dest="identifyPlugin", action="store_true",
+                                help="Attempt to identify what plugins the target is using")
+
+        # obfuscation options
+        anon = OptionGroup(parser, "Anonymity arguments",
+                           "Arguments that help with anonymity and hiding identity")
+
+        anon.add_option("--proxy", dest="proxyConfig", metavar="PROXY-STRING",
+                        help="Use a proxy to do the scraping, will not auto configure to the API's")
+
+        anon.add_option("--proxy-file", dest="proxyFileRand", metavar="FILE-PATH",
+                        help="Grab a random proxy from a given file of proxies")
+
+        anon.add_option("--random-agent", dest="useRandomAgent", action="store_true",
+                        help="Use a random user-agent from the etc/agents.txt file")
+
+        anon.add_option("--agent", dest="usePersonalAgent", metavar="USER-AGENT",
+                        help="Use your own personal user-agent"),
+
+        anon.add_option("--tor", dest="useTor", action="store_true",
+                        help="Use Tor connection as the proxy and set the firefox browser settings to mimic Tor")
+
+        # miscellaneous options
+        misc = OptionGroup(parser, "Misc Options",
+                           "These options affect how the program will run")
+
+        misc.add_option("--verbose", dest="runInVerbose", action="store_true",
+                        help="Run the application in verbose mode (more output)")
+
+        misc.add_option("--batch", dest="runInBatch", action="store_true",
+                        help="Skip the questions and run in default batch mode")
+
+        misc.add_option("--update", dest="updateZeus", action="store_true",
+                        help="Update to the latest development version")
+
+        misc.add_option("--hide", dest="hideBanner", action="store_true",
+                        help="Hide the banner during running")
+
+        misc.add_option("--version", dest="showCurrentVersion", action="store_true",
+                        help="Show the current version and exit")
+
+        # being worked on
+        # TODO:/
+        misc.add_option("-T", "--x-threads", dest="amountOfThreads", metavar="THREAD-AMOUNT", type=int,
+                        help=SUPPRESS_HELP)
+
+        misc.add_option("--show-success", dest="showSuccessRate", action="store_true",
+                        help="Calculate the dorks success rate and output the calculation in human readable form")
+
+        misc.add_option("--show-description", dest="showPluginDescription", action="store_true",
+                        help="Show the description of the identified plugins")
+
+        parser.add_option_group(mandatory)
+        parser.add_option_group(attacks)
+        parser.add_option_group(search_items)
+        parser.add_option_group(anon)
+        parser.add_option_group(engines)
+        parser.add_option_group(misc)
+
+        opt, _ = parser.parse_args()
+        return opt
+
+    @staticmethod
+    def single_show_args(opt):
+        """
+        parses Zeus's single time run arguments
+        """
+        if opt.showCurrentVersion:
+            print(lib.core.settings.VERSION_STRING)
+            exit(0)
+        if opt.showSqlmapArguments:
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "there are a total of {} arguments understood by sqlmap API, "
+                "they include:".format(len(lib.attacks.sqlmap_scan.sqlmap_opts.SQLMAP_API_OPTIONS))
+            ))
+            print("\n")
+            for arg in lib.attacks.sqlmap_scan.sqlmap_opts.SQLMAP_API_OPTIONS:
+                print(
+                    "[*] {}".format(arg)
+                )
+            print("\n")
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "for more information about sqlmap arguments, see here '{}'".format(
+                    lib.core.settings.SQLMAP_MAN_PAGE_URL
+                )
+            ))
+            lib.core.common.shutdown()
+
+        if opt.showNmapArgs:
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "there are a total of {} arguments understood by nmap, they include:".format(
+                    len(lib.attacks.nmap_scan.nmap_opts.NMAP_API_OPTS)
+                )
+            ))
+            print("\n")
+            for arg in lib.attacks.nmap_scan.nmap_opts.NMAP_API_OPTS:
+                print(
+                    "[*] {}".format(arg)
+                )
+            print("\n")
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "for more information on what the arguments do please see here '{}'".format(
+                    lib.core.settings.NMAP_MAN_PAGE_URL
+                )
+            ))
+            lib.core.common.shutdown()
+
+        # update the program
+        if opt.updateZeus:
+            lib.core.settings.logger.info(lib.core.settings.set_color(
+                "update in progress"
+            ))
+            lib.core.settings.update_zeus()
+            lib.core.common.shutdown()
+
+    @staticmethod
+    def verify_args(args=sys.argv):
+        not_implemented_args = (
+            "-T", "--x-threads", "--force-ssl", "--thread",
+            "-g", "--github-search", "-u", "--url"
+        )
+        # check if any of the arguments are not implemented that have been passed
+        # via the command line
+        # TODO:/
+        # need to create a way to parse all arguments for compatibility with one another
+        for arg in args:
+            for nia in not_implemented_args:
+                if arg == nia:
+                    raise lib.core.errors.ZeusArgumentException(
+                        "\n\nit appears that one of the arguments you have passed ('{}'), "
+                        "has not been implemented into Zeus production yet. This usually means "
+                        "that the option is still in testing and is not ready for use. Arguments "
+                        "that are still in testing are: {}\n".format(
+                            nia, ", ".join(["'{}'".format(a) for a in not_implemented_args])
+                        )
+                    )

lib/core/settings.py

@@ -8,13 +8,11 @@ import shlex
 import difflib
 import logging
 import base64
-import string
 import random
 import socket
 import struct
 import platform
 import subprocess
-
 try:
     import ConfigParser  # python 2
 except ImportError:
@@ -46,7 +44,7 @@ CLONE = "https://github.com/ekultek/zeus-scanner.git"
 ISSUE_LINK = "https://github.com/ekultek/zeus-scanner/issues"
 
 # current version <major.minor.commit.patch ID>
-VERSION = "1.3".format(PATCH_ID)
+VERSION = "1.5.2.{}".format(PATCH_ID)
 
 # colors to output depending on the version
 VERSION_TYPE_COLORS = {"dev": 33, "stable": 92, "other": 30}
@@ -64,10 +62,9 @@ SAYING = "Advanced Reconnaissance..."
 
 # i had to create a banner because something not so good happened...
 DISCLAIMER = (
-    "[!] legal disclaimer: Usage of Zeus for exploiting, blackmailing, or attacking targets without "
+    "[!] legal disclaimer: Usage of Zeus for attacking targets without prior mutual consent is illegal. "
-    "mutual consent is illegal. It is the end users responsibility to obey all laws, regulations, and "
+    "It is the end user's responsibility to obey all applicable local, state and federal laws. "
-    "rules set forth by their state, country, or place of origin. Zeus developers take no responsibility "
+    "Developers assume no liability and are not responsible for any misuse or damage caused by this program."
-    "for any misuse or damage caused by the use of this program."
 )
 
 # sexy banner
@@ -176,9 +173,15 @@ BLACKLIST_FILE_PATH = "{}/log/blacklist".format(os.getcwd())
 # found PGP keys file path
 PGP_KEYS_FILE_PATH = "{}/log/pgp_keys".format(os.getcwd())
 
+# found sqli sites file path
+SQLI_SITES_FILEPATH = "{}/log/sqli-sites".format(os.getcwd())
+
 # the current log file being used
 CURRENT_LOG_FILE_PATH = "{}/log".format(os.getcwd())
 
+# nmap scan log path
+NMAP_LOG_FILE_PATH = "{}/log/nmap-scan-log".format(os.getcwd())
+
 # filename for sitemap log file
 SITEMAP_FILENAME = "{}-sitemap.xml"
 
@@ -188,11 +191,14 @@ ROBOTS_TXT_FILENAME = "{}-robots_text.log"
 # filename for found admin pages log file
 ADMIN_PAGE_FILENAME = "{}-admin-page.log"
 
+# sites found to be possible SQL injection vulnerable
+SQLI_FOUND_FILENAME = "sqli-sites.log"
+
 # filename for clickjacking log file
 CLICKJACKING_FILENAME = "{}-clickjacking.html"
 
 # filename for gists log file
-GIST_FILENAME = "gist-match-{}.log"
+GIST_FILENAME = "{}-gist-match.log"
 
 # filename for whois lookup log file
 WHOIS_LOOKUP_FILENAME = "{}-whois.json"
@@ -222,7 +228,7 @@ BLACKLIST_FILENAME = ".blacklist"
 BLACKWIDOW_FILENAME = "blackwidow-log-{}.log"
 
 # filename for nmap scans
-NMAP_FILENAME = "nmap-scan-results-{}.json"
+NMAP_FILENAME = "{}-nmap-scan-results.json"
 
 # github autohorization token path
 GITHUB_AUTH_PATH = "{}/etc/auths/git_auth".format(os.getcwd())
@@ -264,6 +270,9 @@ PROTECTION_CHECK_PAYLOAD = (
 # scripts to detect the WAF/IDS/IPS
 DETECT_FIREWALL_PATH = "{}/lib/firewall".format(os.getcwd())
 
+# path to run the plugins detection scripts
+DETECT_PLUGINS_PATH = "{}/lib/plugins".format(os.getcwd())
+
 # search engines that the application can use
 AUTHORIZED_SEARCH_ENGINES = {
     "aol": "http://aol.com",
@@ -274,13 +283,6 @@ AUTHORIZED_SEARCH_ENGINES = {
     "pgp": "https://pgp.mit.edu/pks/lookup?search={}&op=index"
 }
 
-# search page for Gists and rate checking URL
-GITHUB_GIST_SEARCH_URLS = {
-    "search": "https://api.github.com/gists/public?page={}&per_page=100",
-    "check_rate": "https://api.github.com/users/ZeusIssueReporter"
-
-}
-
 # extensions to exclude from the spider
 SPIDER_EXT_EXCLUDE = (
     "3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "adp", "ai", "aif", "aiff",
@@ -306,9 +308,11 @@ URL_EXCLUDES = (
     "drive.google", "books.google", "news.google",
     "www.google", "mail.google", "accounts.google",
     "schema.org", "www.<b", "https://cid-", "https://<strong",  # these are some weird things that get pulled up?
-    "plus.google", "www.w3.org", "schemas.live.com",
+    "plus.google", "www.w3.org", "schemas.live.com", "https://my."
     "torproject.org", "search-results.com", "index.com",
-    "gov", ".gov", "facebook.com", "instagram.com", "snapchat"
+    "gov", ".gov", "facebook.com", "instagram.com", "snapchat",
+    "stackoverflow", "stackexchange", "github.com", "apple.com",
+    "http://my.", "root.cern"
 )
 
 # regular expressions used for DBMS recognition based on error message response
@@ -331,14 +335,16 @@ DBMS_ERRORS = {
 
 
 # this has to be the first function so that I can use it in the logger settings below
-def create_log_name(log_path="{}/log", filename="zeus-log-{}.log", matcher="zeus-log"):
+def create_log_name(log_path="{}/log", filename="zeus-log-{}.log", matcher="zeus"):
     """
     create the current log file name by figuring out how many files are there
     """
     if not os.path.exists(log_path.format(os.getcwd())):
         os.mkdir(log_path.format(os.getcwd()))
-    find_file_amount = len([f for f in os.listdir(log_path.format(os.getcwd())) if matcher in f])
+    find_file_amount = len(
-    full_log_path = "{}/{}".format(log_path.format(os.getcwd()), filename.format(find_file_amount + 1))
+        [f for f in os.listdir(log_path.format(os.getcwd())) if matcher in f and not os.path.isdir(f)]
+    ) + 1
+    full_log_path = "{}/{}".format(log_path.format(os.getcwd()), filename.format(find_file_amount))
     return full_log_path
 
 
@@ -451,7 +457,7 @@ def setup(verbose=False):
     """
     if verbose:
         logger.debug(set_color(
-            "checking if the application has been run before...", level=10
+            "checking if the application has been run before", level=10
         ))
     bin.unzip_gecko.main(verbose=verbose)
 
@@ -499,12 +505,12 @@ def grab_random_agent(agent_path="{}/etc/text_files/agents.txt", verbose=False):
     """
     if verbose:
         logger.debug(set_color(
-            "grabbing random user-agent from '{}'...".format(agent_path.format(os.getcwd())), level=10
+            "grabbing random user-agent from '{}'".format(agent_path.format(os.getcwd())), level=10
         ))
     with open(agent_path.format(os.getcwd())) as agents:
         retval = random.choice(agents.readlines())
     logger.info(set_color(
-        "random agent being used '{}'...".format(retval.strip())
+        "random agent being used '{}'".format(retval.strip())
     ))
     return retval.strip()
 
@@ -541,7 +547,7 @@ def update_zeus():
         return os.system("git pull origin master")
     else:
         logger.fatal(set_color(
-            "no git repository found in directory, unable to update automatically..."
+            "no git repository found in directory, unable to update automatically"
         ))
 
 
@@ -594,28 +600,34 @@ def search_for_process(name):
     return False if not any(name in proc for proc in list(all_process_names)) else True
 
 
-def get_browser_version():
+def get_browser_version(output=True):
     """
     obtain the firefox browser version, this is necessary because zeus can only handle certain versions.
     """
+    if output:
         logger.info(set_color(
-        "attempting to get firefox browser version..."
+            "attempting to get firefox browser version"
         ))
     try:
         firefox_version_command = shlex.split("firefox --version")
         output = subprocess.check_output(firefox_version_command)
-    except (OSError, Exception):
+    except OSError:
         logger.error(set_color(
-            "failed to run firefox...", level=50
+            "failed to run firefox", level=50
         ))
         return "failed to start"
     try:
         major, minor = map(int, re.search(r"(\d+).(\d+)", output).groups())
-    except (ValueError, Exception):
+    except ValueError:
         logger.error(set_color(
-            "failed to parse '{}' for version number...".format(output), level=50
+            "failed to parse '{}' for version number".format(output), level=50
         ))
-        return "failed to gather"
+        return output
+    except Exception as e:
+        logger.error(set_color(
+            "received and exception from firefox '{}'".format(str(e), level=50)
+        ))
+        return str(e)
     return major, minor
 
 
@@ -629,11 +641,15 @@ def config_headers(**kwargs):
     rand_agent = kwargs.get("rand_agent", None)
     verbose = kwargs.get("verbose", False)
     if proxy is not None:
+        if "127.0.0.1" in proxy:
+            logger.warning(set_color(
+                "timeout will be increased to 40s due to Tor being used", level=30
+            ))
         proxy_retval = proxy
     elif rand_proxy is not None:
         if verbose:
             logger.debug(set_color(
-                "loading random proxy from '{}'...".format(rand_proxy), level=10
+                "loading random proxy from '{}'".format(rand_proxy), level=10
             ))
         with open(rand_proxy) as proxies:
             possible = proxies.readlines()
@@ -659,14 +675,14 @@ def get_md5sum(url="https://raw.githubusercontent.com/Ekultek/Zeus-Scanner/maste
         return True
 
 
-def create_identifier(chars=string.ascii_letters):
+def create_identifier(st):
     """
     create the identifier for your Github issue
     """
-    retval = []
+    import hashlib
-    for _ in range(0, 7):
+    obj = hashlib.md5()
-        retval.append(random.choice(chars))
+    obj.update(st)
-    return "".join(retval)
+    return obj.hexdigest()[1:9]
 
 
 def config_search_engine(**kwargs):
@@ -679,8 +695,8 @@ def config_search_engine(**kwargs):
     ddg = kwargs.get("ddg", False)
     enum = kwargs.get("enum", None)
 
-    non_default_msg = "specified to use non-default search engine..."
+    non_default_msg = "specified to use non-default search engine"
-    se_message = "using '{}' as the search engine..."
+    se_message = "using '{}' as the search engine"
     if ddg:
         if verbose:
             logger.debug(set_color(
@@ -692,7 +708,7 @@ def config_search_engine(**kwargs):
         se = AUTHORIZED_SEARCH_ENGINES["duckduckgo"]
     elif aol:
         logger.warning(set_color(
-            "AOL will take a little longer due to pop-ups...", level=30
+            "AOL will take a little longer due to pop-ups", level=30
         ))
         if verbose:
             logger.debug(set_color(
@@ -713,16 +729,16 @@ def config_search_engine(**kwargs):
         se = AUTHORIZED_SEARCH_ENGINES["bing"]
     elif enum is not None:
         logger.info(set_color(
-            "running enumeration on given file '{}'...".format(enum)
+            "running enumeration on given file '{}'".format(enum)
         ))
         se = None
     else:
         if verbose:
             logger.debug(set_color(
-                "using default search engine (Google)...", level=10
+                "using default search engine (Google)", level=10
             ))
         logger.info(set_color(
-            "using default search engine..."
+            "using default search engine"
         ))
         se = AUTHORIZED_SEARCH_ENGINES["google"]
     return se
@@ -739,7 +755,7 @@ def create_arguments(**kwargs):
     conf_file = kwargs.get("conf", None)
 
     logger.info(set_color(
-        "creating arguments for {}...".format("sqlmap" if sqlmap else "nmap")
+        "creating arguments for {}".format("sqlmap" if sqlmap else "nmap")
     ))
     retval = []
     splitter = {"sqlmap": ",", "nmap": "|"}
@@ -751,7 +767,7 @@ def create_arguments(**kwargs):
                     if o.lower() == opt[0]:
                         retval.append((o, opt[1]))
     elif sqlmap:
-        warn_msg = "option '{}' is not recognized by sqlmap API, skipping..."
+        warn_msg = "option '{}' is not recognized by sqlmap API, skipping"
         if sqlmap_args is not None:
             for line in sqlmap_args.split(splitter["sqlmap"]):
                 try:
@@ -774,7 +790,7 @@ def create_arguments(**kwargs):
                         ))
 
     elif nmap:
-        warning_msg = "option {} is not known by the nmap api, skipping..."
+        warning_msg = "option {} is not known by the nmap api, skipping"
         if nmap_args is not None:
             for line in nmap_args.split(splitter["nmap"]):
                 try:
@@ -812,7 +828,7 @@ def create_random_ip():
     if generated == "0.0.0.0" or "255.255.255.255":
         generated = __get_nodes()  # if it isn't a real IP regenerate it
     logger.info(set_color(
-        "random IP address generated for header '{}'...".format(generated)
+        "random IP address generated for header '{}'".format(generated)
     ))
     return generated
 
@@ -840,7 +856,7 @@ def check_for_protection(protected, attack_type):
 
         if attack_type in items or "all" in items:
             logger.warning(set_color(
-                "provided target seems to have protection against this attack type...", level=30
+                "provided target seems to have protection against this attack type", level=30
             ))
             protected.clear()  # clear the set
         return True
@@ -854,7 +870,7 @@ def deprecation(target_version, method, connect=True, *args, **kwargs):
         print(
             "[{} DEPRECATION] {}".format(
                 time.strftime("%H:%M:%S"), set_color(
-                    "{} will be deprecated by version {}...".format(
+                    "{} will be deprecated by version {}".format(
                         method.__name__, target_version
                     ), level=35
                 )
@@ -866,7 +882,7 @@ def deprecation(target_version, method, connect=True, *args, **kwargs):
             "[{} DEPRECATION] {}".format(
                 time.strftime("%H:%M:%S"), set_color(
                     "{} has been deprecated and will no longer work, "
-                    "this attack type will be completely removed by v{}...".format(
+                    "this attack type will be completely removed by v{}".format(
                         method.__name__, target_version
                     ), level=35
                 )
@@ -882,10 +898,10 @@ def check_thread_num(number, batch=False, default=5):
     logger.warning(set_color(
         "you have specified {} threads, it is highly advised to not go over {} threads, "
         "doing so will most likely not give a significant performance increase and also "
-        "will most likely cause unforeseen issues...".format(number, MAX_THREADS), level=30
+        "will most likely cause unforeseen issues".format(number, MAX_THREADS), level=30
     ))
     question_msg = "would you like to continue anyways"
-    default_msg = "defaulting to 5 threads..."
+    default_msg = "defaulting to 5 threads"
     if not batch:
         question = lib.core.common.prompt(
             question_msg, opts="yN"
@@ -917,7 +933,7 @@ def run_attacks(url, **kwargs):
     verbose = kwargs.get("verbose", False)
     whois = kwargs.get("whois", False)
     clickjacking = kwargs.get("clickjacking", False)
-    github = kwargs.get("github", False)
+    # github = kwargs.get("github", False)
     pgp = kwargs.get("pgp", False)
     auto_start = kwargs.get("auto_start", False)
     sqlmap_arguments = kwargs.get("sqlmap_args", None)
@@ -932,6 +948,7 @@ def run_attacks(url, **kwargs):
     agent = kwargs.get("agent", None)
     conf_file = kwargs.get("conf_file", None)
     threads = kwargs.get("threads", None)
+    force_ssl = kwargs.get("ssl", False)
 
     if threads > MAX_THREADS:
         threads = check_thread_num(threads, batch=batch)
@@ -955,7 +972,7 @@ def run_attacks(url, **kwargs):
                 "as of now only 1 attack is supported at a time, choose "
                 "your attack and try again. You can use the -f flag if "
                 "you do not want to complete an entire search again "
-                "(IE -f /home/me/zeus-scanner/log/url-log/url-log-1.log)...", level=40
+                "(IE -f /home/me/zeus-scanner/log/url-log/url-log-1.log)", level=40
             ))
             lib.core.common.shutdown()
 
@@ -979,7 +996,7 @@ def run_attacks(url, **kwargs):
             from lib.attacks import nmap_scan
             url_ip_address = replace_http(url.strip())
             return nmap_scan.perform_port_scan(
-                url_ip_address, verbose=verbose,
+                url_ip_address, verbose=verbose, timeout=timeout,
                 opts=create_arguments(nmap=True, nmap_args=nmap_arguments)
             )
         elif admin:
@@ -994,6 +1011,7 @@ def run_attacks(url, **kwargs):
                 main_xss(
                     url, verbose=verbose, proxy=proxy,
                     agent=agent, tamper=tamper_script, batch=batch,
+                    force_ssl=force_ssl
                 )
         elif whois:
             from lib.attacks.whois_lookup.whois import whois_lookup_main
@@ -1005,10 +1023,10 @@ def run_attacks(url, **kwargs):
             if check_for_protection(PROTECTED, "clickjacking"):
                 clickjacking_main(url, agent=agent, proxy=proxy,
                                   forward=forwarded, batch=batch)
-        elif github:
+        # elif github:
-            from lib.attacks.gist_lookup import github_gist_search_main
+        #    from lib.attacks.gist_lookup import github_gist_search_main
-            query = replace_http(url)
+        #    query = replace_http(url)
-            github_gist_search_main(query, agent=agent, proxy=proxy, verbose=verbose)
+        #    github_gist_search_main(query, agent=agent, proxy=proxy, verbose=verbose)
         elif pgp:
             from var.search.pgp_search import pgp_main
             pgp_main(url, verbose=verbose)
@@ -1016,7 +1034,7 @@ def run_attacks(url, **kwargs):
             pass
     else:
         logger.warning(set_color(
-            "skipping '{}'...".format(url), level=30
+            "skipping '{}'".format(url), level=30
         ))
 
 
@@ -1042,7 +1060,7 @@ def parse_blacklist(dork, path, batch=False):
                 if not question.lower().startswith("y"):
                     lib.core.common.shutdown()
             else:
-                lib.core.common.prompt(prompt_msg, opts="yN", default="y")
+                lib.core.common.prompt(prompt_msg, opts="yN", default="n")
     return True
 
 
@@ -1099,7 +1117,7 @@ def get_token(path):
     return token
 
 
-def tails(file_object, last_lines=35):
+def tails(file_object, last_lines=50):
     """
     return the last `n` lines of a file, much like the Unix
     tails command
@@ -1117,3 +1135,11 @@ def tails(file_object, last_lines=35):
                 lines = list(file_object)
             pos *= 2
     return "".join(lines[-last_lines:])
+
+
+def convert_to_minutes(seconds):
+    """
+    convert an amount of seconds to minutes and seconds
+    """
+    import time
+    return time.strftime("%M:%S", time.gmtime(seconds))

lib/firewall/akamai.py

@@ -0,0 +1,21 @@
+import re
+
+from lib.core.common import HTTP_HEADER
+
+
+__item__ = "AkamaiGHost Website Protection (Akamai Global Host)"
+
+
+def detect(content, **kwargs):
+    headers = kwargs.get("headers", None)
+    content = str(content)
+    detection_schema = (
+        re.compile(r"you.don.t.have.permission.to.access", re.I),
+        re.compile(r"<.+>access.denied<.+.>", re.I),
+    )
+    for detection in detection_schema:
+        if detection.search(content) is not None:
+            if re.compile(r"\bakamaighost", re.I).search(headers.get(HTTP_HEADER.SERVER, "")) is not None:
+                return True
+            if re.compile(r"\bak.bmsc.", re.I).search(headers.get(HTTP_HEADER.SET_COOKIE, "")) is not None:
+                return True

lib/firewall/armor.py

@@ -0,0 +1,15 @@
+import re
+
+
+__item__ = "Armor Protection (Armor Defense)"
+
+
+def detect(content, **kwargs):
+    content = str(content)
+    detection_schema = (
+        re.compile(r"\barmor\b", re.I),
+        re.compile(r"blocked.by.website.protection.from.armour", re.I)
+    )
+    for detection in detection_schema:
+        if detection.search(content) is not None:
+            return True

lib/firewall/dw.py

@@ -0,0 +1,16 @@
+import re
+
+
+__item__ = "DynamicWeb Injection Check (DynamicWeb)"
+
+
+def detect(content, **kwargs):
+    headers = kwargs.get("headers", None)
+    status = kwargs.get("status", None)
+    detection_schema = (
+        re.compile(r"dw.inj.check", re.I),
+    )
+    if status == 403:
+        for detection in detection_schema:
+            if detection.search(headers.get("X-403-status-by", "")) is not None:
+                return True

lib/firewall/fortigate.py

@@ -0,0 +1,22 @@
+import re
+
+from lib.core.common import HTTP_HEADER
+
+
+__item__ = "FortiWeb Web Application Firewall (Fortinet)"
+
+
+def detect(content, **kwargs):
+    headers = kwargs.get("headers", None)
+    content = str(content)
+    detection_schema = (
+        re.compile(r"<.+>powered.by.fortinet<.+.>", re.I),
+        re.compile(r"<.+>fortigate.ips.sensor<.+.>", re.I),
+        re.compile(r"fortigate", re.I), re.compile(r".fgd_icon", re.I),
+        re.compile(r"\AFORTIWAFSID=", re.I)
+    )
+    for detection in detection_schema:
+        if detection.search(content) is not None:
+            return True
+        if detection.search(headers.get(HTTP_HEADER.SET_COOKIE, "")) is not None:
+            return True

lib/firewall/generic.py

@@ -26,8 +26,8 @@ def detect(content, **kwargs):
     detection_schema = (
         re.compile("blocked", re.I), re.compile("forbidden", re.I),
         re.compile("illegal", re.I), re.compile("reported", re.I),
-        re.compile("logged", re.I), re.compile("access denied", re.I),
+        re.compile("ip.logged", re.I), re.compile("access.denied", re.I),
-        re.compile("ip address logged", re.I), re.compile("banner", re.I),
+        re.compile("ip.address.logged", re.I), re.compile(r"not.acceptable")
     )
     for detection in detection_schema:
         if detection.search(content) is not None:

lib/header_check/__init__.py

@@ -1,25 +1,26 @@
 import os
 import re
+import time
 import importlib
 import unicodedata
 
-import requests
 from xml.dom import minidom
-from requests.exceptions import ConnectionError
+from requests.exceptions import (
+    ConnectionError,
+    ReadTimeout
+)
 
 from var.auto_issue.github import request_issue_creation
-from lib.core.cache import cache
 from lib.core.common import (
     write_to_log_file,
     shutdown,
     pause,
-    HTTP_HEADER
+    get_page,
+    HTTP_HEADER,
 )
 from lib.core.settings import (
     logger, set_color,
     HEADER_XML_DATA,
-    proxy_string_to_dict,
-    create_random_ip,
     replace_http,
     HEADER_RESULT_PATH,
     COOKIE_LOG_PATH,
@@ -30,55 +31,42 @@ from lib.core.settings import (
     UNKNOWN_FIREWALL_FINGERPRINT_PATH,
     UNKNOWN_FIREWALL_FILENAME,
     COOKIE_FILENAME,
-    HEADERS_FILENAME
+    HEADERS_FILENAME,
+    SQLI_FOUND_FILENAME,
+    SQLI_SITES_FILEPATH,
+    DETECT_PLUGINS_PATH
 )
 
 
-@cache
+def get_charset(html, headers, **kwargs):
-def detect_protection(url, **kwargs):
+    """
-    verbose = kwargs.get("verbose", False)
+    detect the target URL charset
-    agent = kwargs.get("agent", None)
+    """
-    proxy = kwargs.get("proxy", None)
+    charset_regex = re.compile(r'charset=[\"]?([a-zA-Z0-9_-]+)', re.I)
-    xforward = kwargs.get("xforward", False)
+    charset = charset_regex.search(html)
-
+    if charset is not None:
-    if xforward:
+        return charset.group(1)
-        ip_list = (
-            create_random_ip(),
-            create_random_ip(),
-            create_random_ip()
-        )
-        headers = {
-            HTTP_HEADER.CONNECTION: "close",
-            HTTP_HEADER.USER_AGENT: agent,
-            HTTP_HEADER.X_FORWARDED_FOR: "{}, {}, {}".format(ip_list[0], ip_list[1], ip_list[2])
-        }
     else:
-        headers = {
+        content = headers.get(HTTP_HEADER.CONTENT_TYPE, "")
-            HTTP_HEADER.CONNECTION: "close",
+        charset = charset_regex.search(content)
-            HTTP_HEADER.USER_AGENT: agent
+        if charset is not None:
-        }
+            return charset.group(1)
+    return None
 
-    url = "{} {}".format(url.strip(), PROTECTION_CHECK_PAYLOAD)
 
-    if verbose:
+def detect_protection(url, status, html, headers, **kwargs):
-        logger.debug(set_color(
+    verbose = kwargs.get("verbose", False)
-            "attempting connection to '{}'...".format(url), level=10
-        ))
     try:
-        protection_check_req = requests.get(
-            url, params=headers, proxies=proxy_string_to_dict(proxy), timeout=20
-        )
-
-        html, status, headers = protection_check_req.content, protection_check_req.status_code, protection_check_req.headers
-
         # make sure there are no DBMS errors in the HTML
         for dbms in DBMS_ERRORS:
             for regex in DBMS_ERRORS[dbms]:
                 if re.compile(regex).search(html) is not None:
                     logger.warning(set_color(
                         "it appears that the WAF/IDS/IPS check threw a DBMS error and may be vulnerable "
-                        "to SQL injection attacks. it appears the backend DBMS is '{}'...".format(dbms), level=30
+                        "to SQL injection attacks. it appears the backend DBMS is '{}', site will be "
+                        "saved for further processing".format(dbms), level=30
                     ))
+                    write_to_log_file(url, SQLI_SITES_FILEPATH, SQLI_FOUND_FILENAME)
                     return None
 
         retval = []
@@ -87,7 +75,7 @@ def detect_protection(url, **kwargs):
             item = item[:-3]
             if verbose:
                 logger.debug(set_color(
-                    "loading script '{}'...".format(item), level=10
+                    "loading script '{}'".format(item), level=10
                 ))
             detection_name = "lib.firewall.{}"
             detection_name = detection_name.format(item)
@@ -100,16 +88,18 @@ def detect_protection(url, **kwargs):
                     del retval[retval.index("Generic (Unknown)")]
                 except (Exception, IndexError):
                     logger.warning(set_color(
-                        "multiple firewalls identified ({}), displaying most likely...".format(
+                        "multiple firewalls identified ({}), displaying most likely".format(
-                            ", ".join(retval)
+                            ", ".join([item.split("(")[0] for item in retval])
                         ), level=30
                     ))
                     del retval[retval.index(retval[1])]
+                    if len(retval) >= 2:
+                        del retval[retval.index(retval[1])]
             if retval[0] == "Generic (Unknown)":
                 logger.warning(set_color(
                     "discovered firewall is unknown to Zeus, saving fingerprint to file. "
                     "if you know the details or the context of the firewall please create "
-                    "an issue ({}) with the fingerprint, or a pull request with the script...".format(
+                    "an issue ({}) with the fingerprint, or a pull request with the script".format(
                         ISSUE_LINK
                     ), level=30
                 ))
@@ -122,19 +112,54 @@ def detect_protection(url, **kwargs):
             return None
 
     except Exception as e:
-        if "Read timed out." or "Connection reset by peer" in str(e):
+        if any(err in str(e) for err in ["Read timed out.", "Connection reset by peer"]):
             logger.warning(set_color(
-                "detection request failed, assuming no protection and continuing...", level=30
+                "detection request failed, assuming no protection and continuing", level=30
             ))
             return None
         else:
             logger.exception(set_color(
-                "Zeus ran into an unexpected error '{}'...".format(e), level=50
+                "Zeus ran into an unexpected error '{}'".format(e), level=50
             ))
             request_issue_creation()
             return None
 
 
+def detect_plugins(html, headers, **kwargs):
+    verbose = kwargs.get("verbose", False)
+
+    try:
+        retval = []
+        plugin_skip_schema = ("__init__", ".pyc")
+        plugin_file_list = [f for f in os.listdir(DETECT_PLUGINS_PATH) if not any(s in f for s in plugin_skip_schema)]
+        for plugin in plugin_file_list:
+            plugin = plugin[:-3]
+            if verbose:
+                logger.debug(set_color(
+                    "loading script '{}'".format(plugin), level=10
+                ))
+            plugin_detection = "lib.plugins.{}"
+            plugin_detection = plugin_detection.format(plugin)
+            plugin_detection = importlib.import_module(plugin_detection)
+            if plugin_detection.search(html, headers=headers) is True:
+                retval.append((plugin_detection.__product__, plugin_detection.__description__))
+        if len(retval) > 0:
+            return retval
+        return None
+    except Exception as e:
+        logger.exception(str(e))
+        if "Read timed out." or "Connection reset by peer" in str(e):
+            logger.warning(set_color(
+                "plugin request failed, assuming no plugins and continuing", level=30
+            ))
+            return None
+        else:
+            logger.exception(set_color(
+                "plugin detection has failed with error {}".format(str(e))
+            ))
+            request_issue_creation()
+
+
 def load_xml_data(path, start_node="header", search_node="name"):
     """
     load the XML data
@@ -147,36 +172,15 @@ def load_xml_data(path, start_node="header", search_node="name"):
     return retval
 
 
-def load_headers(url, **kwargs):
+def load_headers(url, req, **kwargs):
     """
     load the HTTP headers
     """
-    agent = kwargs.get("agent", None)
-    proxy = kwargs.get("proxy", None)
-    xforward = kwargs.get("xforward", False)
-
     literal_match = re.compile(r"\\(\X(\d+)?\w+)?", re.I)
 
-    if proxy is not None:
-        proxy = proxy_string_to_dict(proxy)
-    if not xforward:
-        header_value = {
-            HTTP_HEADER.CONNECTION: "close",
-            HTTP_HEADER.USER_AGENT: agent
-        }
-    else:
-        ip_list = create_random_ip(), create_random_ip(), create_random_ip()
-        header_value = {
-            HTTP_HEADER.CONNECTION: "close",
-            HTTP_HEADER.USER_AGENT: agent,
-            HTTP_HEADER.X_FORWARDED_FOR: "{}, {}, {}".format(
-                ip_list[0], ip_list[1], ip_list[2]
-            )
-        }
-    req = requests.get(url, params=header_value, proxies=proxy, timeout=10)
     if len(req.cookies) > 0:
         logger.info(set_color(
-            "found a request cookie, saving to file...", level=25
+            "found a request cookie, saving to file", level=25
         ))
         try:
             cookie_start = req.cookies.keys()
@@ -195,17 +199,28 @@ def load_headers(url, **kwargs):
     http_headers = req.headers
     for header in http_headers:
         try:
+            # check for Unicode in the string, this is just a safety net in case something is missed
+            # chances are nothing will be matched
+            if literal_match.search(header) is not None:
+                retval[header] = unicodedata.normalize(
+                    "NFKD", u"{}".format(http_headers[header])
+                ).encode("ascii", errors="ignore")
+            else:
                 # test to see if there are any unicode errors in the string
-            retval[header] = unicodedata.normalize("NFKD", u"{}".format(http_headers[header])).encode("ascii", errors="ignore")
+                retval[header] = unicodedata.normalize(
+                    "NFKD", u"{}".format(http_headers[header])
+                ).encode("ascii", errors="ignore")
         # just to be safe, we're going to put all the possible Unicode errors into a tuple
         except (UnicodeEncodeError, UnicodeDecodeError, UnicodeError, UnicodeTranslateError, UnicodeWarning):
-            # if there are, we're going to append them to a `do_not_use` list
+            # if there are any errors, we're going to append them to a `do_not_use` list
             do_not_use.append(header)
+    # clear the dict so we can re-add to it
     retval.clear()
     for head in http_headers:
         # if the header is in the list, we skip it
         if head not in do_not_use:
             retval[head] = http_headers[head]
+    # return a dict of safe unicodeless HTTP headers
     return retval
 
 
@@ -228,74 +243,150 @@ def main_header_check(url, **kwargs):
     agent = kwargs.get("agent", None)
     proxy = kwargs.get("proxy", None)
     xforward = kwargs.get("xforward", False)
-    identify = kwargs.get("identify", True)
+    identify_waf = kwargs.get("identify_waf", True)
+    identify_plugins = kwargs.get("identify_plugins", True)
+    show_description = kwargs.get("show_description", False)
+    attempts = kwargs.get("attempts", 3)
 
+    default_sleep_time = 5
     protection = {"hostname": url}
     definition = {
         "x-xss": ("protection against XSS attacks", "XSS"),
         "strict-transport": ("protection against unencrypted connections (force HTTPS connection)", "HTTPS"),
         "x-frame": ("protection against clickjacking vulnerabilities", "CLICKJACKING"),
         "x-content": ("protection against MIME type attacks", "MIME"),
+        "x-csrf": ("protection against Cross-Site Forgery attacks", "CSRF"),
+        "x-xsrf": ("protection against Cross-Site Forgery attacks", "CSRF"),
         "public-key": ("protection to reduce success rates of MITM attacks", "MITM"),
         "content-security": ("header protection against multiple attack types", "ALL")
     }
 
     try:
-        if identify:
+        req, status, html, headers = get_page(url, proxy=proxy, agent=agent, xforward=xforward)
-            logger.info(set_color(
-                "checking if target URL is protected by some kind of WAF/IPS/IDS..."
-            ))
-            identified = detect_protection(url, proxy=proxy, agent=agent, verbose=verbose, xforward=xforward)
 
-            if identified is None:
         logger.info(set_color(
-                    "no WAF/IDS/IPS has been identified on target URL...", level=25
+            "detecting target charset"
+        ))
+        charset = get_charset(html, headers)
+        if charset is not None:
+            logger.info(set_color(
+                "target charset appears to be '{}'".format(charset), level=25
             ))
         else:
             logger.warning(set_color(
-                    "the target URL WAF/IDS/IPS has been identified as '{}'...".format(identified), level=35
+                "unable to detect target charset", level=30
+            ))
+        if identify_waf:
+            waf_url = "{} {}".format(url.strip(), PROTECTION_CHECK_PAYLOAD)
+            _, waf_status, waf_html, waf_headers = get_page(waf_url, xforward=xforward, proxy=proxy, agent=agent)
+            logger.info(set_color(
+                "checking if target URL is protected by some kind of WAF/IPS/IDS"
+            ))
+            if verbose:
+                logger.debug(set_color(
+                    "attempting connection to '{}'".format(waf_url), level=10
+                ))
+
+            identified_waf = detect_protection(url, waf_status, waf_html, waf_headers, verbose=verbose)
+
+            if identified_waf is None:
+                logger.info(set_color(
+                    "no WAF/IDS/IPS has been identified on target URL", level=25
+                ))
+            else:
+                logger.warning(set_color(
+                    "the target URL WAF/IDS/IPS has been identified as '{}'".format(identified_waf), level=35
+                ))
+
+        if identify_plugins:
+            logger.info(set_color(
+                "attempting to identify plugins"
+            ))
+            identified_plugin = detect_plugins(html, headers, verbose=verbose)
+            if identified_plugin is not None:
+                for plugin in identified_plugin:
+                    if show_description:
+                        logger.info(set_color(
+                            "possible plugin identified as '{}' (description: '{}')".format(
+                                plugin[0], plugin[1]
+                            ), level=25
+                        ))
+                    else:
+                        logger.info(set_color(
+                            "possible plugin identified as '{}'".format(
+                                plugin[0]
+                            ), level=25
+                        ))
+            else:
+                logger.warning(set_color(
+                    "no known plugins identified on target", level=30
                 ))
 
         if verbose:
             logger.debug(set_color(
-                "loading XML data...", level=10
+                "loading XML data", level=10
             ))
         comparable_headers = load_xml_data(HEADER_XML_DATA)
         logger.info(set_color(
-            "attempting to get request headers for '{}'...".format(url.strip())
+            "attempting to get request headers for '{}'".format(url.strip())
         ))
         try:
-            found_headers = load_headers(url, proxy=proxy, agent=agent, xforward=xforward)
+            found_headers = load_headers(url, req)
         except (ConnectionError, Exception) as e:
             if "Read timed out." or "Connection reset by peer" in str(e):
                 found_headers = None
             else:
                 logger.exception(set_color(
-                    "Zeus has hit an unexpected error and cannot continue '{}'...".format(e), level=50
+                    "Zeus has hit an unexpected error and cannot continue '{}'".format(e), level=50
                 ))
                 request_issue_creation()
 
         if found_headers is not None:
             if verbose:
                 logger.debug(set_color(
-                    "fetched {}...".format(found_headers), level=10
+                    "fetched {}".format(found_headers), level=10
                 ))
             headers_established = [str(h) for h in compare_headers(found_headers, comparable_headers)]
             for key in definition.iterkeys():
                 if any(key in h.lower() for h in headers_established):
                     logger.warning(set_color(
-                        "provided target has {}...".format(definition[key][0]), level=30
+                        "provided target has {}".format(definition[key][0]), level=30
                     ))
             for key in found_headers.iterkeys():
                 protection[key] = found_headers[key]
             logger.info(set_color(
-                "writing found headers to log file...", level=25
+                "writing found headers to log file", level=25
             ))
             return write_to_log_file(protection, HEADER_RESULT_PATH, HEADERS_FILENAME.format(replace_http(url)))
         else:
             logger.error(set_color(
-                "unable to retrieve headers for site '{}'...".format(url.strip()), level=40
+                "unable to retrieve headers for site '{}'".format(url.strip()), level=40
+            ))
+    except ConnectionError:
+        attempts = attempts - 1
+        if attempts == 0:
+            return False
+        logger.warning(set_color(
+            "target actively refused the connection, sleeping for {}s and retrying the request".format(
+                default_sleep_time
+            ), level=30
+        ))
+        time.sleep(default_sleep_time)
+        main_header_check(
+            url, proxy=proxy, agent=agent, xforward=xforward, show_description=show_description,
+            identify_plugins=identify_plugins, identify_waf=identify_waf, verbose=verbose,
+            attempts=attempts
+        )
+    except ReadTimeout:
+        logger.error(set_color(
+            "meta-data retrieval failed due to target URL timing out, skipping", level=40
         ))
     except KeyboardInterrupt:
         if not pause():
             shutdown()
+    except Exception as e:
+        logger.exception(set_color(
+            "meta-data retrieval failed with unexpected error '{}'".format(
+                str(e)
+            ), level=50
+        ))

lib/plugins/1024.py

@@ -0,0 +1,24 @@
+import re
+
+
+__product__ = "1024-CMS"
+__description__ = (
+    "1024 is one of a few CMS's leading the way with "
+    "the implementation of the AJAX technology into "
+    "all its areas. This includes dynamic administration "
+    "and user interaction. 1024 offers you to ability to "
+    "set up your own community forums, download area, news "
+    "posts, member management and more."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r".1024cms.", re.I),
+        re.compile(r"<.+>powered.by.1024.cms<.+.>", re.I),
+        re.compile(r"1024.cms", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/360.py

@@ -0,0 +1,23 @@
+import re
+
+
+__product__ = "360 Web Manager"
+__description__ = (
+    "1024 is one of a few CMS's leading the way with the "
+    "implementation of the AJAX technology  into all its "
+    "areas. This includes dynamic adminstration and user "
+    "interaction. 1024 offers you to ability to set up your "
+    "own community forums, download area, news posts, member management and more."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"powered.by.360.web.manager", re.I),
+        re.compile(r"360webmanager.software", re.I),
+        re.compile(r"http(s)?.\S{2}(www.)?360webmanager(.com)?", re.I),
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/3com.py

@@ -0,0 +1,25 @@
+import re
+
+
+__product__ = "3COM-NBX"
+__description__ = (
+    "3COM NBX phone system. The NBX NetSet utility is a web "
+    "interface in which you configure and manage the NBX "
+    "system. NBX systems present the NBX NetSet utility "
+    "through an embedded web server that is integrated in system software."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"nbx.netset", re.I),
+        re.compile(r"<.+>nbx.netset<.+.>", re.I),
+        re.compile(r"3com.corporation", re.I),
+        re.compile(r"nbx.corporation", re.I),
+        re.compile(r"http(s)?.//(www.)?nbxhelpdesk.com", re.I),
+        re.compile(r"nbx.help.desk", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/3dcart.py

@@ -0,0 +1,23 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "3dcart"
+__description__ = (
+    "The 3dcart Shopping Cart Software is a complete e-commerce solution for anyone."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r"3dcart.stats", re.I),
+        re.compile(r"/3dvisit/", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.SET_COOKIE, "")) is not None:
+            return True

lib/plugins/4d.py

@@ -0,0 +1,19 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "4D"
+__description__ = (
+    "4D web application deployment server"
+)
+
+
+def search(html, **kwargs):
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r"/^4D_v[\d]{1,2}(_SQL)?\/([\d\.]+)$/", re.I),
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.SERVER, "")) is not None:
+            return True

lib/plugins/4images.py

@@ -0,0 +1,23 @@
+import re
+
+
+__product__ = "4images"
+__description__ = (
+    "4images is a powerful web-based image gallery "
+    "management system. Features include comment system, "
+    "user registration and management, password protected "
+    "administration area with browser-based upload and HTML "
+    "templates for page layout and design."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_protection_schema = (
+        re.compile(r"http(s)?.//(www.)?4homepages.\w+", re.I),
+        re.compile(r"powered.by.<.+>4images<.+.>", re.I),
+        re.compile(r"powered.by.4images", re.I)
+    )
+    for plugin in plugin_protection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/68classified.py

@@ -0,0 +1,19 @@
+import re
+
+
+__product__ = "68-Classifieds-Script"
+__description__ = (
+    "68 Classifieds Script - Requires PHP"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"http(s)?.//(www.)?68classifieds.com", re.I),
+        re.compile(r"68.classifieds.script", re.I),
+        re.compile(r"68.classifieds", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/aardvark.py

@@ -0,0 +1,19 @@
+import re
+
+
+__product__ = "Aardvark-Topsites-PHP"
+__description__ = (
+    "Aardvark Topsites PHP is a free topsites script built on PHP and MySQL"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"powered.by.aardvark.topsites.php", re.I),
+        re.compile(r"aardvark.topsites.php", re.I),
+        re.compile(r"http(s)?.//(www.)?aardvarktopsitesphp.com", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/abyss.py

@@ -0,0 +1,20 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "Abyss-Web-Server"
+__description__ = (
+    "Abyss Web Server is a compact web server available "
+    "for Windows, Mac OS X, Linux, and FreeBSD operating systems"
+)
+
+
+def search(html, **kwargs):
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r"/^Abyss\/([^\s]+)/", re.I),
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.SERVER, "")) is not None:
+            return True

lib/plugins/accellion.py

@@ -0,0 +1,22 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "Accellion-Secure-File-Transfer"
+__description__ = (
+    "Accellion Secure File Transfer (SFT)"
+)
+
+
+def search(html, **kwargs):
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r"/sfcurl.deleted./", re.I),
+        re.compile(r"/\/courier\/[\d]+@\/mail_user_login\.html\?$/", re.I),
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.LOCATION, "")) is not None:
+            return True
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.SET_COOKIE, "")) is not None:
+            return True

lib/plugins/atomfeed.py

@@ -0,0 +1,19 @@
+import re
+
+
+__product__ = "Atom Feed"
+__description__ = (
+    "Atom Feeds allow software programs to check for updates published on a website"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"<link.\w+.[\"]?atom.xml[\"]?", re.I),
+        re.compile(r"type.[\"]?application.atom.xml[\"]?", re.I),
+        re.compile(r"title.[\"]?sitewide.atom.feed[\"]?", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/b2evolution.py

@@ -0,0 +1,21 @@
+import re
+
+
+__product__ = "b2evolution"
+__description__ = (
+    "b2evolution is a powerful blog tool you can install on your own website"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"b2evolution", re.I),
+        re.compile(r"powered.by.b\devolution", re.I),
+        re.compile(r"powered.by.b\devolution.\d{3}\w+.gif", re.I),
+        re.compile(r"http(s)?.//(www.)?b2evolution.net", re.I),
+        re.compile(r"visit.b2evolution.s.website", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/bmcremedy.py

@@ -0,0 +1,16 @@
+import re
+
+
+__product__ = "BMC Remedy"
+__description__ = (
+    "BMC Remedy is an IT management ticketing system designed by BMC Software"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"<.+>bmc.\w+.remedy.\w+.mid.\w+.tier.\w+.\d+.\d+...login<.+.>", re.I),
+        re.compile(r".bmc.remedy.action.request.system.", re.I),
+        re.compile(r"class.[\'\"]?caption[\'\"]?.\W{1,3}\w+..[0-9]{4}.bmc.software[,]?.inc[orporated]?.", re.I)
+    )

lib/plugins/bomgar.py

@@ -0,0 +1,27 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "Bomgar"
+__description__ = (
+    "Bomgar simplifies support by letting technicians control "
+    "remote computers, servers, smartphones and network devices "
+    "over the internet or network. With Bomgar, a support rep can "
+    "see what customers see or control their computers for support"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(".bomgar.", re.I),
+        re.compile(r"http(s)?.//(www.)?bomgar.com", re.I),
+        re.compile(r"alt.[\'\"]?remote.support.by.bomgar[\'\"]?", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.SERVER, "")) is not None:
+            return True
+        if plugin.search(html) is not None:
+            return True

lib/plugins/clipbucket.py

@@ -0,0 +1,21 @@
+import re
+
+
+__product__ = "ClipBucket"
+__description__ = (
+    "ClipBucket is an Open Source and freely downloadable PHP "
+    "script that will let you start your own Video Sharing website"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"<.\S+.clipbucket", re.I),
+        re.compile(r"content.[\'\"]clipbucket", re.I),
+        re.compile(r"http(s)?.//(www.)?clip.bucket.com", re.I),
+        re.compile(r"http(s)?.//(www.)?clipbucket.com", re.I),
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/googleapi.py

@@ -0,0 +1,20 @@
+import re
+
+
+__product__ = "Google API"
+__description__ = (
+    "Google APIs is a set of application programming interfaces (APIs) developed by Google "
+    "which allow communication with Google Services and their integration to other services"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"src.[\'\"]?http(s)?.//googleapis.com", re.I),
+        re.compile(r"src.[\'\"]?http(s)?.//ajax.googleapis.com", re.I),
+        re.compile(r".googleapis.", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/html5.py

@@ -0,0 +1,20 @@
+import re
+
+
+__product__ = "HTML5"
+__description__ = (
+    "HTML5 is a markup language used for structuring and presenting "
+    "content on the World Wide Web. It is the fifth and current major "
+    "version of the HTML standard."
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r".html5.", re.I),
+        re.compile(r"\bhtml\d+", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/ihtml.py

@@ -0,0 +1,25 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "iHTML"
+__description__ = (
+    "iHTML is a server side internet/web programming and scripting "
+    "language in used by thousands of sites worldwide to deliver "
+    "cost effective dynamic database driven web sites"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r".ihtml.", re.I),
+        re.compile(r"\bihtml.", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.X_POWERED_BY, "")) is not None:
+            return True

lib/plugins/jquery.py

@@ -0,0 +1,20 @@
+import re
+
+
+__product__ = "JQuery"
+__description__ = (
+    "A fast, concise, JavaScript that simplifies how to traverse "
+    "HTML documents, handle events, perform animations, and add AJAX"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"src.[\'\"]?http(s)?.//ajax.googleapis.com.ajax.libs.jquery.\d.\d.\d", re.I),
+        re.compile(r".jquery.", re.I),
+        re.compile(r"jquery.min.js", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/moodle.py

@@ -0,0 +1,19 @@
+import re
+
+
+__product__ = "Moodle"
+__description__ = (
+    "Moodle is an opensource educational software written in PHP"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r".moodle.", re.I),
+        re.compile(r".moodlesession.", re.I),
+        re.compile(r".php.moodlesession.(\w+)?(\d+)?", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/mssqlreportmanager.py

@@ -0,0 +1,18 @@
+import re
+
+
+__product__ = "Microsoft SQL Report Manager"
+__description__ = (
+    "Microsoft SQL Server Report Manager - web-based report access and management tool"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"content.[\'\"]?microsoft.sql.server.report", re.I),
+        re.compile(r"microsoft.sql.server.report.manager", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/opengraph.py

@@ -0,0 +1,25 @@
+import re
+
+
+__product__ = "Open-Graph-Protocol"
+__description__ = (
+    "The Open Graph protocol enables you to integrate "
+    "your Web pages into the social graph. It is currently "
+    "designed for Web pages representing profiles of real-world "
+    "things. Things like movies, sports teams, celebrities, "
+    "and restaurants. Including Open Graph tags on your Web page, "
+    "makes your page equivalent to a Facebook Page"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r".og.title.", re.I),
+        re.compile(".fb.admins.", re.I),
+        re.compile(r".og.type.", re.I),
+        re.compile(r".fb.app.id.", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/plugins/openxchange.py

@@ -0,0 +1,24 @@
+import re
+
+import lib.core.common
+
+
+__product__ = "Open-Xchange-Server"
+__description__ = (
+    "Open Xchange Mail Server"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    headers = kwargs.get("headers", None)
+    plugin_detection_schema = (
+        re.compile(r"open.xchange.server", re.I),
+        re.compile(r"javascript.to.access.the.open.xchange.server", re.I),
+        re.compile(r"/^http(s)?://(www.)?[^\/]+\/ox6\/ox\.html$/", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True
+        if plugin.search(headers.get(lib.core.common.HTTP_HEADER.LOCATION, "")) is not None:
+            return True

lib/plugins/rssfeed.py

@@ -0,0 +1,20 @@
+import re
+
+
+__product__ = "RSS Feed"
+__description__ = (
+    "RSS (Rich Site Summary) is a type of web feed which allows "
+    "users to access updates to online content in a standardized, "
+    "computer-readable format"
+)
+
+
+def search(html, **kwargs):
+    html = str(html)
+    plugin_detection_schema = (
+        re.compile(r"type.[\'\"]?application/rss.xml[\'\"]?", re.I),
+        re.compile(r"title.[\'\"]?rss.feed[\'\"]?", re.I)
+    )
+    for plugin in plugin_detection_schema:
+        if plugin.search(html) is not None:
+            return True

lib/tamper_scripts/base64_encode.py

@@ -11,6 +11,6 @@ def tamper(payload, **kwargs):
     if warning:
         logger.warning(set_color(
             "base64 tamper scripts may increase the possibility of not finding vulnerabilities "
-            "in otherwise vulnerable sites...", level=30
+            "in otherwise vulnerable sites", level=30
         ))
     return base64.b64encode(payload)

lib/tamper_scripts/enclosebrackets_encode.py

@@ -12,7 +12,7 @@ def tamper(payload, **kwargs):
     if warning:
         logger.warning(set_color(
             "enclosing brackets is meant to be used as an obfuscation "
-            "against an already valid vulnerable site...", level=30
+            "against an already valid vulnerable site", level=30
         ))
 
     to_enclose = string.digits

lib/tamper_scripts/hex_encode.py

@@ -8,7 +8,7 @@ def tamper(payload, **kwargs):
     warning = kwargs.get("warning", True)
     if warning:
         logger.warning(set_color(
-            "hex tamper scripts may increase the risk of false positives...", level=30
+            "hex tamper scripts may increase the risk of false positives", level=30
         ))
     retval = hex(hash(payload))
     if "-" in str(retval):

lib/tamper_scripts/multispace2comment_encode.py

@@ -0,0 +1,13 @@
+import random
+
+
+def tamper(payload, **kwargs):
+    possible_spaces = [2, 3, 4]
+    retval = ""
+    encoder = "/**/"
+    for char in retval:
+        if char == " ":
+            retval += encoder * random.choice(possible_spaces)
+        else:
+            retval += char
+    return retval

lib/tamper_scripts/obfuscateentity_encode.py

@@ -0,0 +1,29 @@
+from lib.core.settings import (
+    logger,
+    set_color
+)
+
+
+def tamper(payload, **kwargs):
+    warning = kwargs.get("warning", True)
+
+    if warning:
+        logger.warning(set_color(
+            "obfuscating payloads by their entity encoding equivalent may increase the "
+            "risk of false positives", level=30
+        ))
+
+    skip = ";"
+    encoding_schema = {
+        " ": "&nbsp;", "<": "&lt;", ">": "&gt;",
+        "&": "&amp;", '"': "&quot;", "'": "&apos;",
+    }
+    retval = ""
+    for char in str(payload):
+        if char in encoding_schema.iterkeys():
+            retval += encoding_schema[char]
+        elif char not in encoding_schema.iterkeys() and char != skip:
+            retval += char
+        else:
+            retval += char
+    return retval

lib/tamper_scripts/obfuscateordinal_encode.py

@@ -10,7 +10,7 @@ def tamper(payload, **kwargs):
     if warning:
         logger.warning(set_color(
             "obfuscating the payloads by ordinal equivalents may increase the risk "
-            "of false positives...", level=30
+            "of false positives", level=30
         ))
 
     retval = ""

lib/tamper_scripts/space2null_encode.py

@@ -9,7 +9,7 @@ def tamper(payload, **kwargs):
     if warning:
         logger.warning(set_color(
             "NULL encoding tamper scripts may increase the possibility of not finding vulnerabilities "
-            "in otherwise vulnerable sites...", level=30
+            "in otherwise vulnerable sites", level=30
         ))
 
     retval = ""

var/auto_issue/github.py

@@ -40,7 +40,7 @@ def request_issue_creation():
     if not lib.core.settings.get_md5sum():
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
             "it appears that your checksums did not match, therefore it is assumed "
-            "that you have edited some of the code, issue request denied...", level=50
+            "that you have edited some of the code, issue request denied", level=50
         ))
         lib.core.common.shutdown()
 
@@ -49,18 +49,18 @@ def request_issue_creation():
     )
     if question.lower().startswith("n"):
         lib.core.settings.logger.error(lib.core.settings.set_color(
-            "Zeus has experienced an internal error and cannot continue, shutting down...", level=40
+            "Zeus has experienced an internal error and cannot continue, shutting down", level=40
         ))
         lib.core.common.shutdown()
 
     lib.core.settings.fix_log_file()
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "Zeus got an unexpected error and will automatically create an issue for this error, please wait..."
+        "Zeus got an unexpected error and will automatically create an issue for this error, please wait"
     ))
 
     def __extract_stacktrace(file_data):
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "extracting traceback from log file..."
+            "extracting traceback from log file"
         ))
         retval, buff_mode, _buffer = [], False, ""
         with open(file_data, "r+") as log:
@@ -73,19 +73,19 @@ def request_issue_creation():
                     _buffer = ""
                 if buff_mode:
                     if len(line) > 400:
-                        line = line[:400] + "...\n"
+                        line = line[:400] + "\n"
                     _buffer += line
         return "".join(retval)
 
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "getting authorization..."
+        "getting authorization"
     ))
 
     token = lib.core.settings.get_token(lib.core.settings.GITHUB_AUTH_PATH)
 
     current_log_file = lib.core.settings.get_latest_log_file(lib.core.settings.CURRENT_LOG_FILE_PATH)
     stacktrace = __extract_stacktrace(current_log_file)
-    identifier = lib.core.settings.create_identifier()
+    identifier = lib.core.settings.create_identifier(stacktrace)
     issue_title = "Unhandled exception ({})".format(identifier)
     ff_version = lib.core.settings.get_browser_version()
     log_file_information = lib.core.settings.tails(current_log_file)
@@ -95,7 +95,7 @@ def request_issue_creation():
         "body": "Zeus version:\n`{}`\n\n"
                 "Firefox version:\n`{}`\n\n"
                 "Geckodriver version:\n`{}`\n\n"
-                "Error info:\n```{}````\n\n"
+                "Error info:\n```{}```\n\n"
                 "Running details:\n`{}`\n\n"
                 "Commands used:\n`{}`\n\n"
                 "Log file info:\n```{}```".format(
@@ -121,12 +121,12 @@ def request_issue_creation():
         urllib2.urlopen(req, timeout=10).read()
         lib.core.settings.logger.info(lib.core.settings.set_color(
             "issue has been created successfully with the following name '{}', your unique identifier "
-            "for this issue is '{}' and the URL to your issue is '{}'...".format(
+            "for this issue is '{}' and the URL to your issue is '{}'".format(
                 issue_title, identifier, find_url(identifier)
             )
         ))
     except Exception as e:
         lib.core.settings.logger.exception(lib.core.settings.set_color(
             "failed to auto create the issue, got exception '{}', "
-            "you may manually create an issue...".format(e), level=50
+            "you may manually create an issue".format(e), level=50
         ))

var/blackwidow/__init__.py

@@ -1,9 +1,5 @@
 import os
-import sys
-reload(sys)
-sys.setdefaultencoding("utf-8")  # this will take care of most of the Unicode errors.
 
-import requests
 from bs4 import BeautifulSoup
 
 import lib.core.errors
@@ -21,19 +17,8 @@ class Blackwidow(object):
     def __init__(self, url, user_agent=None, proxy=None, forward=None):
         self.url = url
         self.forward = forward or None
-        self.proxy = lib.core.settings.proxy_string_to_dict(proxy) or None
+        self.proxy = proxy
         self.user_agent = user_agent or lib.core.settings.DEFAULT_USER_AGENT
-        if self.forward is not None:
-            self.headers = {
-                "user-agent": self.user_agent,
-                "X-Forwarded-For": "{}, {}, {}".format(
-                    self.forward[0], self.forward[1], self.forward[2]
-                )
-            }
-        else:
-            self.headers = {
-                "user-agent": self.user_agent
-            }
 
     @staticmethod
     def get_url_ext(url):
@@ -51,11 +36,15 @@ class Blackwidow(object):
         make sure the connection is good before you continue
         """
         try:
-            # verify=False will take care of SSLErrors
+            # we'll skip SSL verification to avoid any SSLErrors that might
-            attempt = requests.get(self.url, params=self.headers, proxies=self.proxy, verify=False)
+            # arise, we won't really need it with this anyways
-            if attempt.status_code == 200:
+            attempt, status, _, _ = lib.core.common.get_page(
-                return ("ok", None)
+                self.url, agent=self.user_agent, xforward=self.forward, skip_verf=True,
-            return ("fail", attempt.status_code)
+                proxy=self.proxy
+            )
+            if status == 200:
+                return "ok", None
+            return "fail", attempt.status_code
         except Exception as e:
             if "Max retries exceeded with url" in str(e):
                 info_msg = ""
@@ -64,14 +53,14 @@ class Blackwidow(object):
                 else:
                     info_msg += ""
                 lib.core.settings.logger.fatal(lib.core.settings.set_color(
-                    "provided website '{}' is refusing connection{}...".format(
+                    "provided website '{}' is refusing connection{}".format(
                         self.url, info_msg
                     ), level=50
                 ))
                 lib.core.common.shutdown()
             else:
                 lib.core.settings.logger.exception(lib.core.settings.set_color(
-                    "failed to connect to '{}' received error '{}'...".format(
+                    "failed to connect to '{}' received error '{}'".format(
                         self.url, e
                     ), level=50
                 ))
@@ -84,8 +73,9 @@ class Blackwidow(object):
         """
         unique_links = set()
         true_url = lib.core.settings.replace_http(given_url)
-        req = requests.get(given_url, params=self.headers, proxies=self.proxy)
+        _, status, html_page, _ = lib.core.common.get_page(
-        html_page = req.content
+            given_url, agent=self.user_agent, proxy=self.proxy
+        )
         soup = BeautifulSoup(html_page, "html.parser")
         for link in soup.findAll(attribute):
             found_redirect = str(link.get(descriptor)).decode("unicode_escape")
@@ -114,27 +104,27 @@ def blackwidow_main(url, **kwargs):
 
     if verbose:
         lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "settings user-agent to '{}'...".format(agent), level=10
+            "settings user-agent to '{}'".format(agent), level=10
         ))
     if proxy is not None:
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "running behind proxy '{}'...".format(proxy), level=10
+                "running behind proxy '{}'".format(proxy), level=10
             ))
     lib.core.settings.create_dir("{}/{}".format(os.getcwd(), "log/blackwidow-log"))
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "starting blackwidow on '{}'...".format(url)
+        "starting blackwidow on '{}'".format(url)
     ))
     crawler = Blackwidow(url, user_agent=agent, proxy=proxy, forward=forward)
     if verbose:
         lib.core.settings.logger.debug(lib.core.settings.set_color(
-            "testing connection to the URL...", level=10
+            "testing connection to the URL", level=10
         ))
     test_code = crawler.test_connection()
     if not test_code[0] == "ok":
         error_msg = (
             "connection test failed with status code: {}, reason: '{}'. "
-            "test connection needs to pass, try a different link..."
+            "test connection needs to pass, try a different link"
         )
         for error_code in lib.core.common.STATUS_CODES.keys():
             if error_code == test_code[1]:
@@ -152,15 +142,15 @@ def blackwidow_main(url, **kwargs):
         lib.core.common.shutdown()
     else:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "connection test succeeded, continuing...", level=25
+            "connection test succeeded, continuing", level=25
         ))
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "crawling given URL '{}' for links...".format(url)
+        "crawling given URL '{}' for links".format(url)
     ))
     found = crawler.scrape_page_for_links(url)
     if len(found) > 0:
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "found a total of {} links from given URL '{}'...".format(
+            "found a total of {} links from given URL '{}'".format(
                 len(found), url
             ), level=25
         ))
@@ -168,5 +158,5 @@ def blackwidow_main(url, **kwargs):
                                           filename=lib.core.settings.BLACKWIDOW_FILENAME)
     else:
         lib.core.settings.logger.fatal(lib.core.settings.set_color(
-            "did not find any usable links from '{}'...".format(url), level=50
+            "did not find any usable links from '{}'".format(url), level=50
         ))

var/search/__init__.py

@@ -128,7 +128,7 @@ class SetBrowser(object):
         try:
             if not self.tor:
                 logger.info(set_color(
-                    "setting the browser..."
+                    "setting the browser"
                 ))
                 profile.set_preference("general.useragent.override", self.agent)
                 browser = webdriver.Firefox(profile, proxy=self.__set_proxy())
@@ -137,7 +137,7 @@ class SetBrowser(object):
                 browser = webdriver.Firefox(profile, proxy=self.__set_proxy())
             else:
                 logger.info(set_color(
-                    "setting the Tor browser emulation..."
+                    "setting the Tor browser emulation"
                 ))
                 profile = self.__tor_browser_emulation(profile)
                 browser = webdriver.Firefox(profile)

var/search/pgp_search.py

@@ -109,22 +109,23 @@ def get_pgp_keys(url_list, query, attribute="pre", **kwargs):
     identity_matcher = re.compile(r"\bbegin.pgp.public.key.block", re.I)
     amount_left = len(url_list)
     lib.core.settings.logger.info(lib.core.settings.set_color(
-        "checking a maximum of {} PGP keys...".format(amount_to_search)
+        "checking a maximum of {} PGP keys".format(amount_to_search)
     ))
     for i, url in enumerate(url_list, start=1):
         if i >= amount_to_search:
             break
         if verbose:
             lib.core.settings.logger.debug(lib.core.settings.set_color(
-                "checking '{}'...".format(url), level=10
+                "checking '{}'".format(url), level=10
             ))
         if i % 25 == 0:
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "currently checking PGP key #{}, {} left to check ({} total found)...".format(
+                "currently checking PGP key #{}, {} left to check ({} total found)".format(
                     i, amount_to_search - i, amount_left
                 )
             ))
         identifiers.append(lib.core.settings.PGP_IDENTIFIER_REGEX.search(str(url)).group())
+        try:
             req = requests.get(
                 url,
                 params=__set_headers(agent=agent, xforward=xforward),
@@ -137,6 +138,10 @@ def get_pgp_keys(url_list, query, attribute="pre", **kwargs):
                 context = soup.findAll(attribute)[0]
                 if identity_matcher.search(str(context)) is not None:
                     extracted_keys.add(context)
+        except ReadTimeout:
+            lib.core.settings.logger.error(lib.core.settings.set_color(
+                "PGP key failed connection, assuming no good and skipping", level=40
+            ))
     for i, k in enumerate(extracted_keys):
         pgp_key = str(k).split("<{}>".format(attribute))  # split the string by the tag
         pgp_key = pgp_key[1].split("</{}>".format(attribute))[0]  # split it again by the end tag
@@ -159,33 +164,33 @@ def pgp_main(query, verbose=False):
         except Exception:
             query = query
         lib.core.settings.logger.info(lib.core.settings.set_color(
-            "searching public PGP files with given query '{}'...".format(query)
+            "searching public PGP files with given query '{}'".format(query)
         ))
         try:
             html = obtain_html(
                 lib.core.settings.AUTHORIZED_SEARCH_ENGINES["pgp"], query, agent=lib.core.settings.DEFAULT_USER_AGENT
             )
-        except ReadTimeout:
+        except (Exception, ReadTimeout):
             lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "connection timed out, assuming no PGP keys...", level=30
+                "connection failed, assuming no PGP keys", level=30
             ))
             html = None
         if html is not None:
             urls = gather_urls(html)
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "found a total of {} URLs...".format(len(urls))
+                "found a total of {} URLs".format(len(urls))
             ))
             if verbose:
                 lib.core.settings.logger.debug(lib.core.settings.set_color(
-                    "found a '{}'...".format(urls), level=10
+                    "found a '{}'".format(urls), level=10
                 ))
             lib.core.settings.logger.info(lib.core.settings.set_color(
-                "gathering PGP key(s) and writing to a file...", level=25
+                "gathering PGP key(s) and writing to a file", level=25
             ))
             return get_pgp_keys(urls, query, verbose=verbose)
         else:
             lib.core.settings.logger.warning(lib.core.settings.set_color(
-                "did not find anything using query '{}'...".format(query), level=30
+                "did not find anything using query '{}'".format(query), level=30
             ))
     except KeyboardInterrupt:
         if not lib.core.common.pause():

var/search/selenium_search.py

@@ -1,7 +1,5 @@
 import os
 import time
-import shlex
-import subprocess
 
 try:
     from urllib import (  # python 2
@@ -15,6 +13,7 @@ except ImportError:
 import requests
 from bs4 import BeautifulSoup
 from pyvirtualdisplay import Display
+from requests.exceptions import ConnectionError
 from selenium.webdriver.common.keys import Keys
 from selenium.webdriver.remote.errorhandler import (
     UnexpectedAlertPresentException,
@@ -28,7 +27,8 @@ from lib.core.common import (
     HTTP_HEADER,
     URLParser,
     shutdown,
-    prompt
+    prompt,
+    run_fix
 )
 from lib.core.settings import (
     logger,
@@ -75,20 +75,20 @@ def get_urls(query, url, verbose=False, **kwargs):
     batch = kwargs.get("batch", False)
     xforward = kwargs.get("xforward", False)
     logger.info(set_color(
-        "setting up virtual display to hide the browser..."
+        "setting up virtual display to hide the browser"
     ))
     ff_display = Display(visible=0, size=(800, 600))
     ff_display.start()
     browser = var.search.SetBrowser(agent=user_agent, proxy=proxy, tor=tor, xforward=xforward).set_browser()
-    logger.info(set_color("browser will open shortly...", level=25))
+    logger.info(set_color("browser will open shortly", level=25))
     browser.get(url)
     if verbose:
         logger.debug(set_color(
-            "searching search engine for the 'q' element (search button)...", level=10
+            "searching search engine for the 'q' element (search button)", level=10
         ))
     search = browser.find_element_by_name('q')
     logger.info(set_color(
-        "searching search engine using query '{}'...".format(url, query)
+        "searching search engine using query '{}'".format(query)
     ))
     try:
         # enter the text you want to search and hit enter
@@ -98,31 +98,35 @@ def get_urls(query, url, verbose=False, **kwargs):
             time.sleep(3)
         else:
             logger.warning(set_color(
-                "sleep time has been increased to 10 seconds due to tor being used...", level=30
+                "sleep time has been increased to 10 seconds due to tor being used", level=30
             ))
             time.sleep(10)
     except ElementNotInteractableException:
         # get rid of the popup box and hit enter after entering the text to search
-        browser.execute_script("document.querySelectorAll('label.boxed')[1].click()")
+        try:
+            for _ in range(5):
+                browser.execute_script("document.querySelectorAll('label.boxed')[{}].click()".format(_))
                 search.send_keys(query)
                 search.send_keys(Keys.RETURN)
                 time.sleep(3)
+        except Exception:
+            pass
     except UnicodeDecodeError:
         logger.error(set_color(
             "your query '{}' appears to have unicode characters in it, selenium is not "
-            "properly formatted to handle unicode characters, this dork will be skipped...".format(
+            "properly formatted to handle unicode characters, this dork will be skipped".format(
                 query
             ), level=40
         ))
     if verbose:
         logger.debug(set_color(
-            "obtaining URL from selenium..."
+            "obtaining URL from selenium"
         ))
     try:
         retval = browser.current_url
     except UnexpectedAlertPresentException:
         logger.warning(set_color(
-            "alert present, closing...", level=30
+            "alert present, closing", level=30
         ))
         # discover the alert and close it before continuing
         alert = browser.switch_to.alert
@@ -131,7 +135,7 @@ def get_urls(query, url, verbose=False, **kwargs):
     # if you have been IP banned, we'll extract the URL from it
     if IP_BAN_REGEX.search(retval) is not None:
         logger.warning(set_color(
-            "it appears that Google is attempting to block your IP address, attempting bypass...", level=30
+            "it appears that Google is attempting to block your IP address, attempting bypass", level=30
         ))
         try:
             retval = URLParser(retval).extract_ip_ban_url()
@@ -147,7 +151,7 @@ def get_urls(query, url, verbose=False, **kwargs):
                 )
             else:
                 do_continue = prompt(
-                    question_msg, opts="yN", default="n"
+                    question_msg, opts="yN", default="y"
                 )
 
             # shutdown and write the URL to a file
@@ -155,7 +159,7 @@ def get_urls(query, url, verbose=False, **kwargs):
                 write_to_log_file(retval, EXTRACTED_URL_LOG, EXTRACTED_URL_FILENAME)
                 logger.info(set_color(
                     "it is advised to extract the URL's from the produced URL written to the above "
-                    "(IE open the log, copy the url into firefox)...".format(retval)
+                    "(IE open the log, copy the url into firefox)".format(retval)
                 ))
                 shutdown()
         except Exception as e:
@@ -165,7 +169,7 @@ def get_urls(query, url, verbose=False, **kwargs):
             ff_display.stop()
             logger.exception(set_color(
                 "zeus was unable to extract the correct URL from the ban URL '{}', "
-                "got exception '{}'...".format(
+                "got exception '{}'".format(
                     unquote(retval), e
                 ), level=50
             ))
@@ -173,7 +177,7 @@ def get_urls(query, url, verbose=False, **kwargs):
             shutdown()
     if verbose:
         logger.debug(set_color(
-            "found current URL from selenium browser...", level=10
+            "found current URL from selenium browser", level=10
         ))
     logger.info(set_color(
         "closing the browser and continuing process.."
@@ -201,31 +205,31 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
 
     if verbose:
         logger.debug(set_color(
-            "parsing blacklist...", level=10
+            "parsing blacklist", level=10
         ))
     parse_blacklist(query, BLACKLIST_FILE_PATH, batch=batch)
 
     if verbose:
         logger.debug(set_color(
-            "checking for user-agent and proxy configuration...", level=10
+            "checking for user-agent and proxy configuration", level=10
         ))
 
     if not parse_webcache and "google" in url_to_search:
         logger.warning(set_color(
-            "will not parse webcache URL's (to parse webcache pass -W)...", level=30
+            "will not parse webcache URL's (to parse webcache pass -W)", level=30
         ))
     if not pull_all:
         logger.warning(set_color(
-            "only pulling URLs with GET(query) parameters (to pull all URL's pass -E)...", level=30
+            "only pulling URLs with GET(query) parameters (to pull all URL's pass -E)", level=30
         ))
 
-    user_agent_info = "adjusting user-agent header to {}..."
+    user_agent_info = "adjusting user-agent header to {}"
     if user_agent is not DEFAULT_USER_AGENT:
         user_agent_info = user_agent_info.format(user_agent.strip())
     else:
         user_agent_info = user_agent_info.format("default user agent '{}'".format(DEFAULT_USER_AGENT))
 
-    proxy_string_info = "setting proxy to {}..."
+    proxy_string_info = "setting proxy to {}"
     if proxy_string is not None:
         proxy_string = proxy_string_to_dict(proxy_string)
         proxy_string_info = proxy_string_info.format(
@@ -236,13 +240,13 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
             "tor proxy settings"
         )
     else:
-        proxy_string_info = "no proxy configuration detected..."
+        proxy_string_info = "no proxy configuration detected"
 
     if forward_for:
         ip_to_use = (create_random_ip(), create_random_ip(), create_random_ip())
         if verbose:
             logger.debug(set_color(
-                "random IP addresses generated for headers '{}'...".format(ip_to_use), level=10
+                "random IP addresses generated for headers '{}'".format(ip_to_use), level=10
             ))
 
         headers = {
@@ -256,7 +260,7 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
             HTTP_HEADER.USER_AGENT: user_agent
         }
     logger.info(set_color(
-        "attempting to gather query URL..."
+        "attempting to gather query URL"
     ))
     try:
         query_url = get_urls(
@@ -268,103 +272,84 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
             logger.fatal(set_color(
                 "firefox was not found in the default location on your system, "
                 "check your installation and make sure it is in /usr/lib, if you "
-                "find it there, restart your system and try again...", level=50
+                "find it there, restart your system and try again", level=50
             ))
-        elif "connection refused" in str(e):
+        elif "connection refused" in str(e).lower():
             logger.fatal(set_color(
                 "there are to many sessions of firefox opened and selenium cannot "
-                "create a new one...", level=50
+                "create a new one", level=50
             ))
-            do_autoclean = prompt(
+            run_fix(
-                "would you like to attempt to auto clean the open sessions", opts="yN"
+                "would you like to attempt to auto clean the open sessions",
+                "sudo sh {}".format(CLEANUP_TOOL_PATH),
+                "kill off the open sessions of firefox and re-run Zeus",
+                exit_process=True
             )
-            if do_autoclean.lower().startswith("y"):
-                logger.warning(set_color(
-                    "this will kill all instances of the firefox web browser...", level=30
-                ))
-                auto_clean_command = shlex.split("sudo sh {}".format(CLEANUP_TOOL_PATH))
-                subprocess.call(auto_clean_command)
-                logger.info(set_color(
-                    "all open sessions of firefox killed, it should be safe to re-run "
-                    "Zeus..."
-                ))
-            else:
-                logger.warning(set_color(
-                    "kill off the open sessions of firefox and re-run Zeus...", level=30
-                ))
-            shutdown()
         elif "Program install error!" in str(e):
-            do_fix = prompt(
+            logger.error(set_color(
                 "seems the program is having some trouble installing would you like "
-                "to try and automatically fix this issue", opts="yN"
+                "to try and automatically fix this issue", level=40
+            ))
+            run_fix(
+                "would you like to attempt to fix this issue automatically",
+                "sudo sh {}".format(FIX_PROGRAM_INSTALL_PATH),
+                "you can manually try and re-install Xvfb to fix the problem",
+                exit_process=True
             )
-            if do_fix.lower().startswith("y"):
-                logger.info(set_color(
-                    "attempting to reinstall failing dependency..."
-                ))
-                do_fix_command = shlex.split("sudo sh {}".format(FIX_PROGRAM_INSTALL_PATH))
-                subprocess.call(do_fix_command)
-                logger.info(set_color(
-                    "successfully installed, you should be good to re-run Zeus..."
-                ))
-                shutdown()
-            else:
-                logger.info(set_color(
-                    "you can automatically try and re-install Xvfb to fix the problem..."
-                ))
-                shutdown()
         elif "Message: Reached error page:" in str(e):
             logger.fatal(set_color(
-                "geckodriver has hit an error that usually means it needs to be reinstalled...", level=50
+                "geckodriver has hit an error that usually means it needs to be reinstalled", level=50
             ))
             question = prompt(
                 "would you like to attempt a reinstallation of the geckodriver", opts="yN"
             )
             if question.lower().startswith("y"):
                 logger.warning(set_color(
-                    "rewriting all executed information, path information, and removing geckodriver...", level=30
+                    "rewriting all executed information, path information, and removing geckodriver", level=30
                 ))
                 rewrite_all_paths()
                 logger.info(set_color(
-                    "all paths rewritten, you will be forced to re-install everything next run of Zeus..."
+                    "all paths rewritten, you will be forced to re-install everything next run of Zeus"
                 ))
             else:
                 logger.fatal(set_color(
-                    "you will need to remove the geckodriver from /usr/bin and reinstall it...", level=50
+                    "you will need to remove the geckodriver from /usr/bin and reinstall it", level=50
                 ))
                 shutdown()
         elif "Unable to find a matching set of capabilities" in str(e):
             logger.fatal(set_color(
-                "it appears that firefox, selenium, and geckodriver are not playing nice with one another...", level=50
+                "it appears that firefox, selenium, and geckodriver are not playing nice with one another", level=50
             ))
-            question = prompt(
+            run_fix(
-                "would you like to attempt to resolve this issue automatically", opts="yN"
+                "would you like to attempt to resolve this issue automatically",
+                "sudo sh {}".format(REINSTALL_TOOL),
+                ("you will need to reinstall firefox to a later version, update selenium, and reinstall the "
+                 "geckodriver to continue using Zeus"),
+                exit_process=True
             )
-            if question.lower().startswith("y"):
-                reinstall_command = shlex.split("sudo sh {}".format(REINSTALL_TOOL))
-                subprocess.call(reinstall_command)
-                rewrite_all_paths()
-                shutdown()
-            else:
-                logger.fatal(set_color(
-                    "you will need to reinstall firefox to a later version, update selenium, and reinstall the "
-                    "geckodriver to continue using Zeus...", level=50
-                ))
-                shutdown()
         else:
             logger.exception(set_color(
                 "{} failed to gather the URL from search engine, caught exception '{}' "
-                "exception has been logged to current log file...".format(
+                "exception has been logged to current log file".format(
                     os.path.basename(__file__), str(e).strip()), level=50)
             )
             request_issue_creation()
         shutdown()
     logger.info(set_color(
-        "URL successfully gathered, searching for GET parameters..."
+        "URL successfully gathered, searching for GET parameters"
     ))
 
     logger.info(set_color(proxy_string_info))
+
+    try:
         req = requests.get(query_url, proxies=proxy_string, params=headers)
+    except ConnectionError:
+        logger.warning(set_color(
+            "target machine refused connection, delaying and trying again", level=30
+        ))
+        time.sleep(3)
+        req = requests.get(query_url, proxies=proxy_string, params=headers)
+
     logger.info(set_color(user_agent_info))
     req.headers.update(headers)
     found_urls = URL_REGEX.findall(req.text)
@@ -384,15 +369,15 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
                         if verbose:
                             try:
                                 logger.debug(set_color(
-                                    "found '{}'...".format(url.split(splitter)[0]), level=10
+                                    "found '{}'".format(url.split(splitter)[0]), level=10
                                 ))
                             except TypeError:
                                 logger.debug(set_color(
-                                    "found '{}'...".format(str(url).split(splitter)[0]), level=10
+                                    "found '{}'".format(str(url).split(splitter)[0]), level=10
                                 ))
                             except AttributeError:
                                 logger.debug(set_color(
-                                    "found '{}...".format(str(url)), level=10
+                                    "found '{}".format(str(url)), level=10
                                 ))
                         if url is not None:
                             retval.add(url.split(splitter)[0])
@@ -403,12 +388,12 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
         if parse_webcache:
             if "webcache" in url:
                 logger.info(set_color(
-                    "found a webcache URL, extracting..."
+                    "found a webcache URL, extracting"
                 ))
                 url = URLParser(url).extract_webcache_url()
                 if verbose:
                     logger.debug(set_color(
-                        "found '{}'...".format(url), level=15
+                        "found '{}'".format(url), level=15
                     ))
                 true_retval.add(url)
             else:
@@ -422,18 +407,18 @@ def parse_search_results(query, url_to_search, verbose=False, **kwargs):
             amount_of_urls = len(open(file_path).readlines())
             success_rate = calculate_success(amount_of_urls)
             logger.info(set_color(
-                "provided query has a {} success rate...".format(success_rate)
+                "provided query has a {} success rate".format(success_rate)
             ))
     else:
-        logger.fatal(set_color(
+        logger.warning(set_color(
-            "did not find any URLs with given query '{}' writing query to blacklist...".format(query), level=50
+            "did not find any URLs with given query '{}' writing query to blacklist".format(query), level=50
         ))
         write_to_log_file(query, BLACKLIST_FILE_PATH, BLACKLIST_FILENAME, blacklist=True)
-        shutdown()
+        
     logger.info(set_color(
-        "found a total of {} URLs with given query '{}'...".format(len(true_retval), query)
+        "found a total of {} URLs with given query '{}'".format(len(true_retval), query)
     ))
-    return list(true_retval) if len(true_retval) != 0 else None
+    
 
 
 def search_multiple_pages(query, link_amount, verbose=False, **kwargs):
@@ -450,7 +435,7 @@ def search_multiple_pages(query, link_amount, verbose=False, **kwargs):
     search_engine = AUTHORIZED_SEARCH_ENGINES["search-results"]
 
     logger.warning(set_color(
-        "searching multiple pages will not be done on Google...".format(search_engine), level=30
+        "searching multiple pages will not be done on Google".format(search_engine), level=30
     ))
 
     if not parse_blacklist(query, BLACKLIST_FILE_PATH, batch=batch):
@@ -474,11 +459,11 @@ def search_multiple_pages(query, link_amount, verbose=False, **kwargs):
         while len(retval) <= link_amount:
             if verbose:
                 logger.debug(set_color(
-                    "searching page number {}...".format(page_number), level=10
+                    "searching page number {}".format(page_number), level=10
                 ))
             if page_number % 10 == 0:
                 logger.info(set_color(
-                    "currently on page {} of search results...".format(
+                    "currently on page {} of search results".format(
                         page_number
                     )
                 ))
@@ -500,44 +485,44 @@ def search_multiple_pages(query, link_amount, verbose=False, **kwargs):
                         page_number += 1
                     else:
                         logger.warning(set_color(
-                            "hit max page number {}...".format(MAX_PAGE_NUMBER), level=30
+                            "hit max page number {}".format(MAX_PAGE_NUMBER), level=30
                         ))
                         break
                 else:
                     logger.warning(set_color(
-                        "no more results found for given query '{}'...".format(query), level=30
+                        "no more results found for given query '{}'".format(query), level=30
                     ))
                     break
     except KeyboardInterrupt:
         logger.error(set_color(
-            "user aborted, dumping already found URL(s)...", level=40
+            "user aborted, dumping already found URL(s)", level=40
         ))
         write_to_log_file(retval, URL_LOG_PATH, URL_FILENAME)
         logger.info(set_color(
-            "found a total of {} URL(s)...".format(len(retval)), level=25
+            "found a total of {} URL(s)".format(len(retval)), level=25
         ))
         shutdown()
     except Exception as e:
         logger.exception(set_color(
-            "Zeus ran into an unexpected error '{}'...".format(e), level=50
+            "Zeus ran into an unexpected error '{}'".format(e), level=50
         ))
         request_issue_creation()
         shutdown()
 
     if len(retval) > 0:
         logger.info(set_color(
-            "a total of {} URL(s) found out of the requested {}...".format(len(retval), link_amount), level=25
+            "a total of {} URL(s) found out of the requested {}".format(len(retval), link_amount), level=25
         ))
         file_path = write_to_log_file(retval, URL_LOG_PATH, URL_FILENAME)
         if show_success:
             amount_of_urls = len(open(file_path).readlines())
             success_rate = calculate_success(amount_of_urls)
             logger.info(set_color(
-                "provided query has a {} success rate...".format(success_rate)
+                "provided query has a {} success rate".format(success_rate)
             ))
         return list(retval)
     else:
         logger.warning(set_color(
-            "did not find any links with given query '{}' writing to blacklist...".format(query), level=30
+            "did not find any links with given query '{}' writing to blacklist".format(query), level=30
         ))
         write_to_log_file(query, BLACKLIST_FILE_PATH, BLACKLIST_FILENAME)

zeus.py

@@ -1,10 +1,9 @@
 #!/usr/bin/env python
 
-import os
 import io
+import sys
 import time
 import shlex
-import optparse
 import warnings
 import subprocess
 
@@ -12,12 +11,12 @@ from var import blackwidow
 from var.search import selenium_search
 from var.auto_issue.github import request_issue_creation
 from lib.header_check import main_header_check
-from lib.attacks.nmap_scan.nmap_opts import NMAP_API_OPTS
-from lib.attacks.sqlmap_scan.sqlmap_opts import SQLMAP_API_OPTIONS
 
+from lib.core.parse import ZeusParser
 from lib.core.errors import (
     InvalidInputProvided,
-    InvalidProxyType
+    InvalidProxyType,
+    ZeusArgumentException
 )
 from lib.core.common import (
     start_up,
@@ -26,158 +25,38 @@ from lib.core.common import (
 )
 from lib.core.settings import (
     setup,
-    BANNER,
     logger,
     set_color,
     get_latest_log_file,
-    CURRENT_LOG_FILE_PATH,
-    URL_LOG_PATH,
     get_random_dork,
-    update_zeus,
-    VERSION_STRING,
-    URL_REGEX, URL_QUERY_REGEX,
-    NMAP_MAN_PAGE_URL,
-    SQLMAP_MAN_PAGE_URL,
     fix_log_file,
-    SPIDER_LOG_PATH,
     config_headers,
     config_search_engine,
     find_running_opts,
     run_attacks,
+    CURRENT_LOG_FILE_PATH,
+    SPIDER_LOG_PATH,
+    URL_REGEX, URL_QUERY_REGEX,
+    URL_LOG_PATH,
+    BANNER
 )
 
 warnings.simplefilter("ignore")
 
 if __name__ == "__main__":
 
-    parser = optparse.OptionParser(usage="{} -d|r|l|f|b| DORK|FILE|URL [ATTACKS] [--OPTS]".format(
+    # this will take care of most of the Unicode errors.
-        os.path.basename(__file__)
+    reload(sys)
-    ))
+    sys.setdefaultencoding("utf-8")
+    sys.setrecursionlimit(1500)
 
-    # mandatory options
+    opt = ZeusParser.cmd_parser()
-    mandatory = optparse.OptionGroup(parser, "Mandatory Options",
-                                     "These options have to be used in order for Zeus to run")
-    mandatory.add_option("-d", "--dork", dest="dorkToUse", metavar="DORK",
-                         help="Specify a singular Google dork to use for queries")
-    mandatory.add_option("-l", "--dork-list", dest="dorkFileToUse", metavar="FILE-PATH",
-                         help="Specify a file full of dorks to run through"),
-    mandatory.add_option("-r", "--rand-dork", dest="useRandomDork", action="store_true",
-                         help="Use a random dork from the etc/dorks.txt file to perform the scan")
-    mandatory.add_option("-b", "--blackwidow", dest="spiderWebSite", metavar="URL",
-                         help="Spider a single webpage for all available URL's")
-    mandatory.add_option("-f", "--url-file", dest="fileToEnumerate", metavar="FILE-PATH",
-                         help="Run an attack on URL's in a given file")
 
-    # attack options
+    ZeusParser().single_show_args(opt)
-    attacks = optparse.OptionGroup(parser, "Attack arguments",
-                                   "These arguments will give you the choice on how you want to check the websites")
-    attacks.add_option("-s", "--sqli", dest="runSqliScan", action="store_true",
-                       help="Run a Sqlmap SQLi scan on the discovered URL's")
-    attacks.add_option("-p", "--port-scan", dest="runPortScan", action="store_true",
-                       help="Run a Nmap port scan on the discovered URL's")
-    attacks.add_option("-a", "--admin-panel", dest="adminPanelFinder", action="store_true",
-                       help="Search for the websites admin panel")
-    attacks.add_option("-x", "--xss-scan", dest="runXssScan", action="store_true",
-                       help="Run an XSS scan on the found URL's")
-    attacks.add_option("-w", "--whois-lookup", dest="performWhoisLookup", action="store_true",
-                       help="Perform a WhoIs lookup on the provided domain")
-    attacks.add_option("-c", "--clickjacking", dest="performClickjackingScan", action="store_true",
-                       help="Perform a clickjacking scan on a provided URL")
-    attacks.add_option("-g", "--github-search", dest="searchGithub", action="store_true",
-                       help="Perform a Github Gist search for any information on the found websites")
-    attacks.add_option("-P", "--pgp", dest="pgpLookup", action="store_true",
-                       help="Perform a PGP public key lookup on the found URLs")
-    attacks.add_option("--sqlmap-args", dest="sqlmapArguments", metavar="SQLMAP-ARGS",
-                       help="Pass the arguments to send to the sqlmap API within quotes & "
-                            "separated by a comma. IE 'dbms mysql, verbose 3, level 5'")
-    attacks.add_option("--sqlmap-conf", dest="sqlmapConfigFile", metavar="CONFIG-FILE-PATH",
-                       help="Pass a configuration file that contains the sqlmap arguments")
-    attacks.add_option("--nmap-args", dest="nmapArguments", metavar="NMAP-ARGS",
-                       help="Pass the arguments to send to the nmap API within quotes & "
-                            "separated by a pipe. IE '-O|-p 445, 1080'")
-    attacks.add_option("--show-sqlmap", dest="showSqlmapArguments", action="store_true",
-                       help="Show the arguments that the sqlmap API understands")
-    attacks.add_option("--show-nmap", dest="showNmapArgs", action="store_true",
-                       help="Show the arguments that nmap understands")
-    attacks.add_option("--show-possibles", dest="showAllConnections", action="store_true",
-                       help="Show all connections made during the admin panel search")
-    attacks.add_option("--tamper", dest="tamperXssPayloads", metavar="TAMPER-SCRIPT",
-                       help="Send the XSS payloads through tampering before sending to the target")
-    attacks.add_option("--thread", dest="threadPanels", action="store_true",
-                       help="Run multiple threads on functions that support multi-threading")
-    attacks.add_option("--auto", dest="autoStartSqlmap", action="store_true",
-                       help="Automatically start the sqlmap API (or at least try to)")
 
-    # search engine options
+    # verify all the arguments passed before we continue
-    engines = optparse.OptionGroup(parser, "Search engine arguments",
+    # with the process
-                                   "Arguments to change the search engine used (default is Google)")
+    ZeusParser().verify_args()
-    engines.add_option("-D", "--search-engine-ddg", dest="useDDG", action="store_true",
-                       help="Use DuckDuckGo as the search engine")
-    engines.add_option("-B", "--search-engine-bing", dest="useBing", action="store_true",
-                       help="Use Bing as the search engine")
-    engines.add_option("-A", "--search-engine-aol", dest="useAOL", action="store_true",
-                       help="Use AOL as the search engine")
-
-    # arguments to edit your search patterns
-    search_items = optparse.OptionGroup(parser, "Search options",
-                                        "Arguments that will control the search criteria")
-    search_items.add_option("-L", "--links", dest="amountToSearch", type=int, metavar="HOW-MANY-LINKS",
-                            help="Specify how many links to try and search on Google")
-    search_items.add_option("-M", "--multi", dest="searchMultiplePages", action="store_true",
-                            help="Search multiple pages of Google")
-    search_items.add_option("-E", "--exclude-none", dest="noExclude", action="store_true",
-                            help="Do not exclude URLs because they do not have a GET(query) parameter in them")
-    search_items.add_option("-W", "--webcache", dest="parseWebcache", action="store_true",
-                            help="Parse webcache URLs for the redirect in them")
-    search_items.add_option("--x-forward", dest="forwardedForRandomIP", action="store_true",
-                            help="Add a header called 'X-Forwarded-For' with three random IP addresses")
-    search_items.add_option("--time-sec", dest="controlTimeout", metavar="SECONDS", type=int,
-                            help="Control the sleep time to the WhoIS lookup to prevent errors")
-
-    # obfuscation options
-    anon = optparse.OptionGroup(parser, "Anonymity arguments",
-                                "Arguments that help with anonymity and hiding identity")
-    anon.add_option("--proxy", dest="proxyConfig", metavar="PROXY-STRING",
-                    help="Use a proxy to do the scraping, will not auto configure to the API's")
-    anon.add_option("--proxy-file", dest="proxyFileRand", metavar="FILE-PATH",
-                    help="Grab a random proxy from a given file of proxies")
-    anon.add_option("--random-agent", dest="useRandomAgent", action="store_true",
-                    help="Use a random user-agent from the etc/agents.txt file")
-    anon.add_option("--agent", dest="usePersonalAgent", metavar="USER-AGENT",
-                    help="Use your own personal user-agent"),
-    anon.add_option("--tor", dest="useTor", action="store_true",
-                    help="Use Tor connection as the proxy and set the firefox browser settings to mimic Tor")
-
-    # miscellaneous options
-    misc = optparse.OptionGroup(parser, "Misc Options",
-                                "These options affect how the program will run")
-    misc.add_option("--verbose", dest="runInVerbose", action="store_true",
-                    help="Run the application in verbose mode (more output)")
-    misc.add_option("--batch", dest="runInBatch", action="store_true",
-                    help="Skip the questions and run in default batch mode")
-    misc.add_option("--update", dest="updateZeus", action="store_true",
-                    help="Update to the latest development version")
-    misc.add_option("--hide", dest="hideBanner", action="store_true",
-                    help="Hide the banner during running")
-    misc.add_option("--version", dest="showCurrentVersion", action="store_true",
-                    help="Show the current version and exit")
-    misc.add_option("-T", "--x-threads", dest="amountOfThreads", metavar="THREAD-AMOUNT", type=int,
-                    help="Specify how many threads you want to pass")
-    misc.add_option("--show-success", dest="showSuccessRate", action="store_true",
-                    help="Calculate the dorks success rate and output the calculation in human readable form")
-
-    parser.add_option_group(mandatory)
-    parser.add_option_group(attacks)
-    parser.add_option_group(search_items)
-    parser.add_option_group(anon)
-    parser.add_option_group(engines)
-    parser.add_option_group(misc)
-
-    opt, _ = parser.parse_args()
-
-    if opt.showCurrentVersion:
-        print(VERSION_STRING)
-        exit(0)
 
     # run the setup on the program
     setup(verbose=opt.runInVerbose)
@@ -187,59 +66,14 @@ if __name__ == "__main__":
 
     start_up()
 
-    if opt.showSqlmapArguments:
-        logger.info(set_color(
-            "there are a total of {} arguments understood by sqlmap API, "
-            "they include:".format(len(SQLMAP_API_OPTIONS))
-        ))
-        print("\n")
-        for arg in SQLMAP_API_OPTIONS:
-            print(
-                "[*] {}".format(arg)
-            )
-        print("\n")
-        logger.info(set_color(
-            "for more information about sqlmap arguments, see here '{}'...".format(
-                SQLMAP_MAN_PAGE_URL
-            )
-        ))
-        shutdown()
-
-    if opt.showNmapArgs:
-        logger.info(set_color(
-            "there are a total of {} arguments understood by nmap, they include:".format(
-                len(NMAP_API_OPTS)
-            )
-        ))
-        print("\n")
-        for arg in NMAP_API_OPTS:
-            print(
-                "[*] {}".format(arg)
-            )
-        print("\n")
-        logger.info(set_color(
-            "for more information on what the arguments do please see here '{}'...".format(
-                NMAP_MAN_PAGE_URL
-            )
-        ))
-        shutdown()
-
-    # update the program
-    if opt.updateZeus:
-        logger.info(set_color(
-            "update in progress..."
-        ))
-        update_zeus()
-        shutdown()
-
     if opt.runInVerbose:
         being_run = find_running_opts(opt)
         logger.debug(set_color(
-            "running with options '{}'...".format(being_run), level=10
+            "running with options '{}'".format(being_run), level=10
         ))
 
     logger.info(set_color(
-        "log file being saved to '{}'...".format(get_latest_log_file(CURRENT_LOG_FILE_PATH))
+        "log file being saved to '{}'".format(get_latest_log_file(CURRENT_LOG_FILE_PATH))
     ))
 
 
@@ -260,14 +94,14 @@ if __name__ == "__main__":
 
         if urls_to_use is None:
             logger.error(set_color(
-                "unable to run attacks appears that no file was created for the retrieved data...", level=40
+                "unable to run attacks appears that no file was created for the retrieved data", level=40
             ))
             shutdown()
         options = [
             opt.runSqliScan, opt.runPortScan,
             opt.adminPanelFinder, opt.runXssScan,
             opt.performWhoisLookup, opt.performClickjackingScan,
-            opt.searchGithub, opt.pgpLookup
+            opt.pgpLookup
         ]
         if any(options):
             with open(urls_to_use) as urls:
@@ -275,22 +109,29 @@ if __name__ == "__main__":
                     current = i
                     if "webcache" in url:
                         logger.warning(set_color(
-                            "ran into unexpected webcache URL skipping...", level=30
+                            "ran into unexpected webcache URL skipping", level=30
                         ))
                         current -= 1
                     else:
+                        if not url.strip() == "http://" or url == "https://":
                             logger.info(set_color(
-                            "currently running on '{}' (target #{})...".format(
+                                "currently running on '{}' (target #{})".format(
                                     url.strip(), current
                                 ), level=25
                             ))
                             logger.info(set_color(
-                            "checking for HTTP headers..."
+                                "fetching target meta-data"
                             ))
-                        main_header_check(
+                            identified = main_header_check(
                                 url, verbose=opt.runInVerbose, agent=agent_to_use,
-                            proxy=proxy_to_use, xforward=opt.forwardedForRandomIP
+                                proxy=proxy_to_use, xforward=opt.forwardedForRandomIP,
+                                identify_plugins=opt.identifyPlugin, identify_waf=opt.identifyProtection,
+                                show_description=opt.showPluginDescription
                             )
+                            if not identified:
+                                logger.error(set_color(
+                                    "target is refusing to allow meta-data dumping, skipping", level=40
+                                ))
                             run_attacks(
                                 url.strip(),
                                 sqlmap=opt.runSqliScan, nmap=opt.runPortScan, pgp=opt.pgpLookup,
@@ -302,9 +143,13 @@ if __name__ == "__main__":
                                 show_all=opt.showAllConnections, do_threading=opt.threadPanels,
                                 tamper_script=opt.tamperXssPayloads, timeout=opt.controlTimeout,
                                 proxy=proxy_to_use, agent=agent_to_use, conf_file=opt.sqlmapConfigFile,
-                            threads=opt.amountOfThreads
+                                threads=opt.amountOfThreads, force_ssl=opt.forceSSL
                             )
                             print("\n")
+                        else:
+                            logger.warning(set_color(
+                                "malformed URL discovered, skipping", level=30
+                            ))
 
 
     proxy_to_use, agent_to_use = config_headers(
@@ -321,7 +166,7 @@ if __name__ == "__main__":
         # use a personal dork as the query
         if opt.dorkToUse is not None and not opt.searchMultiplePages:
             logger.info(set_color(
-                "starting dork scan with query '{}'...".format(opt.dorkToUse)
+                "starting dork scan with query '{}'".format(opt.dorkToUse)
             ))
             try:
                 selenium_search.parse_search_results(
@@ -331,21 +176,22 @@ if __name__ == "__main__":
                     show_success=opt.showSuccessRate
                 )
             except InvalidProxyType:
-                supported_proxy_types = ["socks5", "socks4", "https", "http"]
+                supported_proxy_types = ("socks5", "socks4", "https", "http")
                 logger.fatal(set_color(
                     "the provided proxy is not valid, specify the protocol and try again, supported "
-                    "proxy protocols are {} (IE socks5://127.0.0.1:9050)...".format(", ".join(supported_proxy_types)), level=50
+                    "proxy protocols are {} (IE socks5://127.0.0.1:9050)".format(
+                        ", ".join(list(supported_proxy_types))), level=50
                 ))
             except Exception as e:
                 if "Permission denied:" in str(e):
                     logger.fatal(set_color(
                         "your permissions are not allowing Zeus to run, "
-                        "try running me with sudo...", level=50
+                        "try running Zeus with sudo", level=50
                     ))
                     shutdown()
                 else:
                     logger.exception(set_color(
-                        "ran into exception '{}'...".format(e), level=50
+                        "ran into exception '{}'".format(e), level=50
                     ))
                 request_issue_creation()
                 pass
@@ -363,7 +209,7 @@ if __name__ == "__main__":
 
             if dork_to_use is None:
                 logger.warning(set_color(
-                    "there has been no dork to specified to do the searching, defaulting to random dork...", level=30
+                    "there has been no dork to specified to do the searching, defaulting to random dork", level=30
                 ))
                 dork_to_use = get_random_dork()
 
@@ -371,14 +217,14 @@ if __name__ == "__main__":
 
             if opt.amountToSearch is None:
                 logger.warning(set_color(
-                    "did not specify amount of links to find defaulting to 75...", level=30
+                    "did not specify amount of links to find defaulting to 75", level=30
                 ))
                 link_amount_to_search = 75
             else:
                 link_amount_to_search = opt.amountToSearch
 
             logger.info(set_color(
-                "searching Google using dork '{}' for a total of {} links...".format(
+                "searching Google using dork '{}' for a total of {} links".format(
                     dork_to_use, link_amount_to_search
                 )
             ))
@@ -392,16 +238,11 @@ if __name__ == "__main__":
             except Exception as e:
                 if "Error 400" in str(e):
                     logger.fatal(set_color(
-                        "failed to connect to search engine...".format(e), level=50
+                        "failed to connect to search engine".format(e), level=50
-                    ))
-                elif "Error 503" in str(e):
-                    logger.fatal(set_color(
-                        "Google has blocked your IP address from doing anymore searches via API, "
-                        "you can still search using headless browsers (-d <DORK>)...", level=50
                     ))
                 else:
                     logger.exception(set_color(
-                        "failed with unexpected error '{}'...".format(e), level=50
+                        "failed with unexpected error '{}'".format(e), level=50
                     ))
                 shutdown()
 
@@ -413,7 +254,7 @@ if __name__ == "__main__":
                 for dork in dorks.readlines():
                     dork = dork.strip()
                     logger.info(set_color(
-                        "starting dork scan with query '{}'...".format(dork)
+                        "starting dork scan with query '{}'".format(dork)
                     ))
                     try:
                         selenium_search.parse_search_results(
@@ -423,7 +264,7 @@ if __name__ == "__main__":
                         )
                     except Exception as e:
                         logger.exception(set_color(
-                            "ran into exception '{}'...".format(e), level=50
+                            "ran into exception '{}'".format(e), level=50
                         ))
                         request_issue_creation()
                         pass
@@ -435,10 +276,10 @@ if __name__ == "__main__":
             random_dork = get_random_dork().strip()
             if opt.runInVerbose:
                 logger.debug(set_color(
-                    "choosing random dork from etc/dorks.txt...", level=10
+                    "choosing random dork from etc/dorks.txt", level=10
                 ))
             logger.info(set_color(
-                "using random dork '{}' as the search query...".format(random_dork)
+                "using random dork '{}' as the search query".format(random_dork)
             ))
             try:
                 selenium_search.parse_search_results(
@@ -450,7 +291,7 @@ if __name__ == "__main__":
 
             except Exception as e:
                 logger.exception(set_color(
-                    "ran into exception '{}' and cannot continue, saved to current log file...".format(e),
+                    "ran into exception '{}' and cannot continue, saved to current log file".format(e),
                     level=50
                 ))
                 request_issue_creation()
@@ -460,7 +301,7 @@ if __name__ == "__main__":
         elif opt.spiderWebSite:
             problem_identifiers = ["http://", "https://"]
             if not URL_REGEX.match(opt.spiderWebSite):
-                err_msg = "URL did not match a true URL{}..."
+                err_msg = "URL did not match a true URL{}"
                 if not any(m in opt.spiderWebSite for m in problem_identifiers):
                     err_msg = err_msg.format(" issue seems to be that http:// "
                                              "or https:// is not present in the URL")
@@ -496,7 +337,7 @@ if __name__ == "__main__":
         # enumerate a file and run attacks on the URL's provided
         elif opt.fileToEnumerate is not None:
             logger.info(set_color(
-                "found a total of {} URL's to enumerate in given file...".format(
+                "found a total of {} URL's to enumerate in given file".format(
                     len(open(opt.fileToEnumerate).readlines())
                 )
             ))
@@ -504,7 +345,7 @@ if __name__ == "__main__":
 
         else:
             logger.critical(set_color(
-                "failed to provide a mandatory argument, you will be redirected to the help menu...", level=50
+                "failed to provide a mandatory argument, you will be redirected to the help menu", level=50
             ))
             time.sleep(2)
             zeus_help_menu_command = shlex.split("python zeus.py --help")
@@ -512,7 +353,7 @@ if __name__ == "__main__":
     except IOError as e:
         if "Invalid URL" in str(e):
             logger.exception(set_color(
-                "URL provided is not valid, schema appears to be missing...", level=50
+                "URL provided is not valid, schema appears to be missing", level=50
             ))
             request_issue_creation()
             shutdown()
@@ -523,38 +364,40 @@ if __name__ == "__main__":
             ))
             shutdown()
         elif "No such file or directory" in str(e):
-            logger.exception(e)
             logger.fatal(set_color(
-                "provided file does not exist, make sure you have the full path...", level=50
+                "provided file does not exist, make sure you have the full path", level=50
             ))
+            shutdown()
         else:
             logger.exception(set_color(
-                "Zeus has hit an unexpected error and cannot continue, error code '{}'...".format(e), level=50
+                "Zeus has hit an unexpected error and cannot continue, error code '{}'".format(e), level=50
             ))
             request_issue_creation()
     except KeyboardInterrupt:
         logger.fatal(set_color(
-            "user aborted process...", level=50
+            "user aborted process", level=50
         ))
         shutdown()
     except UnboundLocalError:
         logger.warning(set_color(
             "do not interrupt the browser when selenium is running, "
-            "it will cause Zeus to crash...", level=30
+            "it will cause Zeus to crash", level=30
         ))
+    except ZeusArgumentException:
+        shutdown()
     except Exception as e:
         if "url did not match a true url" in str(e).lower():
             logger.error(set_color(
                 "you did not provide a URL that is capable of being processed, "
                 "the URL provided to the spider needs to contain protocol as well "
                 "ie. 'http://google.com' (it is advised not to add the GET parameter), "
-                "fix the URL you want to scan and try again...", level=40
+                "fix the URL you want to scan and try again", level=40
             ))
             shutdown()
         elif "Service geckodriver unexpectedly exited" in str(e):
             logger.fatal(set_color(
                 "it seems your firefox version is not compatible with the geckodriver version, "
-                "please re-install Zeus and try again...", level=50
+                "please re-install Zeus and try again", level=50
             ))
             shutdown()
         elif "Max retries exceeded with url" in str(e):
@@ -566,7 +409,7 @@ if __name__ == "__main__":
             shutdown()
         else:
             logger.exception(set_color(
-                "ran into exception '{}' exception has been saved to log file...".format(e), level=50
+                "ran into exception '{}' exception has been saved to log file".format(e), level=50
             ))
             request_issue_creation()