Full Code of daniloaleixo/bovespaStockRatings for AI

Repository: daniloaleixo/bovespaStockRatings
Branch: master
Commit: 97845a531dfb
Files: 40
Total size: 68.5 KB

Directory structure:
gitextract_nx7a1d76/

├── .gitignore
├── .gitmodules
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── Dockerfile
├── LICENSE
├── Pipfile
├── Procfile
├── Procfile.windows
├── README.md
├── app.json
├── client/
│   ├── getFromFirebase.js
│   └── styles.css
├── docker-compose.yml
├── functions/
│   ├── .gitignore
│   ├── package.json
│   ├── src/
│   │   └── index.ts
│   ├── tsconfig.json
│   └── tslint.json
├── fundamentus.py
├── gettingstarted/
│   ├── __init__.py
│   ├── settings.py
│   ├── static/
│   │   └── humans.txt
│   ├── urls.py
│   └── wsgi.py
├── hello/
│   ├── __init__.py
│   ├── admin.py
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   ├── models.py
│   ├── templates/
│   │   ├── base.html
│   │   ├── db.html
│   │   └── index.html
│   ├── tests.py
│   └── views.py
├── index.html
├── manage.py
├── requirements.txt
├── services.sh
└── waitingbar.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
__pycache__/*
scripts/__pycache__/*
firebase.json
.env
Pipfile.lock

================================================
FILE: .gitmodules
================================================
[submodule "fundamentalAnalysisHistoricalDataClient"]
	path = fundamentalAnalysisHistoricalDataClient
	url = https://github.com/daniloaleixo/fundamentalAnalysisHistoricalDataClient.git
	branch = master
[submodule "fundamentalAnalysisHistoricalDataServer"]
	path = fundamentalAnalysisHistoricalDataServer
	url = https://github.com/daniloaleixo/fundamentalAnalysisHistoricalDataServer.git
	branch = master


================================================
FILE: CODE_OF_CONDUCT.md
================================================
# Contributor Covenant Code of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

## Our Standards

Examples of behavior that contributes to creating a positive environment include:

* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

## Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at danilo_aleixo@hotmail.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]

[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/


================================================
FILE: CONTRIBUTING.md
================================================
# Contributing

When contributing to this repository, please first discuss the change you wish to make via issue,
email, or any other method with the owners of this repository before making a change. 

Please note we have a code of conduct, please follow it in all your interactions with the project.

## Pull Request Process

1. Ensure any install or build dependencies are removed before the end of the layer when doing a 
   build.
2. Update the README.md with details of changes to the interface, this includes new environment 
   variables, exposed ports, useful file locations and container parameters.
3. Increase the version numbers in any examples files and the README.md to the new version that this
   Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you 
   do not have permission to do that, you may request the second reviewer to merge it for you.

## Code of Conduct

### Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.

### Our Standards

Examples of behavior that contributes to creating a positive environment
include:

* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
  address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
  professional setting

### Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

### Scope

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.

### Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

### Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at [http://contributor-covenant.org/version/1/4][version]

[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/


================================================
FILE: Dockerfile
================================================
FROM python


COPY fundamentus.py /data/fundamentus.py
COPY waitingbar.py /data/waitingbar.py
WORKDIR /data

# RUN pip install -r requirements.txt
RUN pip install --upgrade lxml && \
    pip install --upgrade python_jwt && \
    pip install --upgrade gcloud && \
    pip install --upgrade sseclient && \
    pip install --upgrade Crypto && \
    pip install --upgrade pycryptodome==3.4.3 && \
    pip install --upgrade requests_toolbelt && \
    pip install --upgrade pymongo && \
    pip install sendgrid && \
    python -m pip install pymongo[srv]

CMD python3 fundamentus.py

================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2016 phoemur

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: Pipfile
================================================
[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true


[packages]

django = "*"
gunicorn = "*"
django-heroku = "*"


[requires]

python_version = "3.6"


================================================
FILE: Procfile
================================================
web: gunicorn gettingstarted.wsgi


================================================
FILE: Procfile.windows
================================================
web: python manage.py runserver 0.0.0.0:5000


================================================
FILE: README.md
================================================
# Bovespa Stock Ratings

Projeto de análise de dados fundamentalistas da Bovespa

**If you like this project you can support me.**  
<div>
<a href="https://apoia.se/daniloaleixo-bovespa" target="_blank"><img src="https://i.imgur.com/mFoBjIN.png" alt="Apoia.se" style="height: 51px !important;width: 217px !important;" ></a>
</div>


## Sobre 
O projeto consiste na primeira fonte de informaçôes open-source sobre análise fundamentalista das ações da Bovespa.
O objetivo é juntar informaçôes para tomadas de decisões.

## Estrutura do projeto

O projeto consiste de 4 projetos em repositórios separados:
* [Crawler](https://github.com/daniloaleixo/bovespaStockRatings): script em python que faz um scrapping em algumas páginas para colher as informações para salvar em um banco de dados Mongo.
* [Backend](https://github.com/daniloaleixo/fundamentalAnalysisHistoricalDataServer): Backend que faz a interface, usando GraphQL entre o banco e o front-end.
* [Front-end](https://github.com/daniloaleixo/fundamentalAnalysisHistoricalDataClient): Visualização de dados da aplicação feita, totalmente em Angular.
* [Agente de trading](https://github.com/daniloaleixo/NeuroEvolutionMarketTrader): Agente usando neuro evolução para comprar e vender ações automaticamente


## Configurando o projeto

  
Clonar o repositório principal
```
git clone --recursive git@github.com:daniloaleixo/bovespaStockRatings.git
```

[Opcional] Se quiser clonar o subrepositórios
```
git submodule update --init --recursive
```
  
## Rodando o crawler

### Rodando da imagem docker oficial
```
docker run -e MONGO_URI=$MONGO_URI daniloaleixo/bovespa-stock-ratings-crawler
```
### Fazendo o build da imagem docker
```
docker build -t bovespa_stock_ratings .
docker run -e MONGO_URI=$MONGO_URI -t bovespa_stock_ratings
```



## Análise 
Estou fazendo uma análise baseada nos princípios fundamentalistas do livro [Investidor Inteligente](https://en.wikipedia.org/wiki/The_Intelligent_Investor) do Benjamin Graham:

* Tamanho Adequado 
  * Patrimônio Líquido maior que R$2bi
* Posição Financeira Forte
  * Liquidez Corrente maior que 1,5
  * Dívida Bruta / Patrimônio Líquido < 50%
* Histórico de Dividendos contínuos por, pelo menos, os últimos 20 anos.
  * Ainda não consegui pegar essa informação, estou colocando só o último DY
  * Ultimo DY maior que 2.5%
* Estabilidade nos Ganhos, Nenhum prejuízo nos últimos 10 anos.
  * Ainda não consegui verificar se teve algum prejuízo
* Crescimento nos Ganhos: 10 anos de crescimento nos lucros-por-ação de, pelo menos,
um terço.
  * Ainda não consegui colocar essa informação, mas tenho só 
  * Crescimento no últimos 5 anos maior que 5%
  * ROE > 20%
* Preço sobre Valor de Mercado: O preço da ação inferior a 1,5 x o valor dos ativos
líquidos.
  * P/VPA < 1.5
* P/L Moderado: O preço da ação inferior a 15x o lucro dos últimos 3 anos
  * P/L < 15
* Teste alternativo: Graham multiplicava o P/L pelo preço sobre o valor de mercado e
verifica se o resultado está abaixo de 22.5
  * P/L x P/VPA < 22.5

## Credits
The project was inicially a contribution to this repostory: https://github.com/phoemur/fundamentus


================================================
FILE: app.json
================================================
{
  "name": "Start on Heroku: Python",
  "description": "A barebones Python app, which can easily be deployed to Heroku.",
  "image": "heroku/python",
  "repository": "https://github.com/heroku/python-getting-started",
  "keywords": ["python", "django" ],
  "addons": [ "heroku-postgresql" ],
  "env": {
    "SECRET_KEY": {
      "description": "The secret key for the Django application.",
      "generator": "secret"
    }
  },
  "environments": {
    "test": {
      "scripts": {
        "test-setup": "python manage.py collectstatic --noinput",
        "test": "python manage.py test"
      }
    }
  }
}


================================================
FILE: client/getFromFirebase.js
================================================
// 
// Get info from frebase and show it on the datatable
// 



var database = firebase.database();
var arrayStocksHistory = [];

var resultadoClone;

var lastUpdate;

var comparadores = {
	patrLiq: { value: 2000000000, checked: 1 },
	liqCorr: { value: 1.5, checked: 1 },
	roe: { value: 20, checked: 1 },
	divPat: { value: 50, checked: 1 },
	cresc: { value: 5, checked: 1 },
	pvp: { value: 1.5, checked: 1 },
	pl: { value: 15, checked: 1 },
	dy: { value: 2.5, checked: 1 },
	plxpvp: { value: 22.5, checked: 1 },
}

initInputs();


// *******************************
// 			Logic functions
// *******************************
database.ref().child('stocks').once('value').then(function(snapshot){
	// console.log(snapshot.val())

	Object.keys(snapshot.val()).forEach(function(key){
		// console.log(snapshot.val()[key])
		var object = JSON.parse(snapshot.val()[key]);
		arrayStocksHistory.push(object)
		// console.log(object);
	})

	// Sort by date, the first one will be the newest
	arrayStocksHistory.sort(function(a,b){
		var date1 = new Date(a.date);
		var date2 = new Date(b.date);
		return date2 - date1;
	})

	// console.log(arrayStocksHistory);
	buildTable(arrayStocksHistory[0]);
	hideLoading();

}, function(error){
	console.log("deu erro");
})

// Get last update
database.ref().child('last_date').once('value').then(function(snapshot){

	lastUpdate = Object.keys(snapshot.val())
	.map(function(key){
		return new Date(snapshot.val()[key]);
	})
	.sort(function(a, b) {
		console.log(new Date(a), new Date(b), new Date(a) > new Date(b));
		return new Date(b) > new Date(a) ? 1 : -1;
	})[0];

	$("#lastUpdate").append(lastUpdate);

}, function(error){
	console.log("deu erro");
})



var buildTable = function(data){
	// console.log('vamos montar a tabela', data);

	// Put the objects in an array
	arrayObjects = [];
	Object.keys(data).forEach(function(key){
		arrayObjects.push(data[key])
	});

	arrayObjects = calculateScores(arrayObjects);

	var resultArray = []; 
	arrayObjects
	.forEach(function(element) {
		var innerObject = {};

		if(Object.keys(element).pop().length > 3) {
			Object.keys(element).map(function(innerKey) {
				innerObject = element[innerKey];
				innerObject['papel'] = innerKey;
			});

			var myArray = [];

			myArray.push(innerObject["papel"]);
			myArray.push(innerObject["nota"]);
			myArray.push(innerObject["cotacao"]);
			myArray.push(innerObject["Pat.Liq"]);
			myArray.push(innerObject["Liq.Corr."]);
			myArray.push(innerObject["ROE"]);
			myArray.push(innerObject["Div.Brut/Pat."]);
			myArray.push(innerObject["Cresc.5a"]);
			myArray.push(innerObject["P/VP"]);
			myArray.push(innerObject["P/L"]);
			myArray.push(innerObject["DY"]);
			myArray.push(innerObject["PSR"]);
			myArray.push(innerObject["P/Ativo"]);
			myArray.push(innerObject["P/Cap.Giro"]);
			myArray.push(innerObject["P/EBIT"]);
			myArray.push(innerObject["P/Ativ.Circ.Liq."]);
			myArray.push(innerObject["EV/EBIT"]);
			myArray.push(innerObject["EBITDA"]);
			myArray.push(innerObject["Mrg.Liq."]);
			myArray.push(innerObject["ROIC"]);
			myArray.push(innerObject["Liq.2m."]);
			resultArray.push(myArray);
			
		}

	});
	$('#resultado').DataTable({ 
		'data': resultArray,
		'order'	: [[ 1, "desc" ]], // Order by score
 		"pageLength": 500, // All stocks in the page
 		"createdRow": function( row, data, dataIndex ) {
 			if(data[1] >= 7.5) {
 				$(row).css('background-color', '#cde0cd')
 			}
 		}
	});

	// var txt = "";
	// for(var i = 0; i < arrayObjects.length; i++) {
	// 	if(typeof(arrayObjects[i]) == "object"){

	// 		Object.keys(arrayObjects[i]).forEach(function(stock){

	// 			var greenLine = " class='green-line'";
	// 			// If we see a stock with score below 7.5 we turnoff green
	// 			if(parseFloat(arrayObjects[i][stock]["nota"]) < 7.5)
	// 				greenLine = '';

	// 			var stockLine = "<tr" + greenLine + ">" + 
	// 							"<td class='stock'>" + stock + "</td>" + 
	// 			            	"<td>" + arrayObjects[i][stock]["nota"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["cotacao"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Pat.Liq"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Liq.Corr."] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["ROE"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Div.Brut/Pat."] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Cresc.5a"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/VP"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/L"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["DY"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["PSR"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/Ativo"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/Cap.Giro"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/EBIT"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["P/Ativ.Circ.Liq."] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["EV/EBIT"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["EBITDA"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Mrg.Liq."] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["ROIC"] + "</td>" +
	// 			            	"<td>" + arrayObjects[i][stock]["Liq.2m."] + "</td>" +
	// 			        		"</tr>";

	// 							txt += stockLine;
	// 		})
	// 	}
	// }
	// $(function(){
	// 	$(".table-body").append(txt);
	// 	console.log('estou aqui');
	// 	// $("#resultado").append('dsajdhayasyusadd');
	// })
}

var calculateScores = function(stockArray){
	console.log('calculando socres');
	let dividePor = 0;
	Object.keys(comparadores).forEach(function(elem){
		if(comparadores[elem].checked) dividePor += 1;
	})
	console.log('dividePor', dividePor);

	for(var i = 0; i < stockArray.length; i++) {
		if(typeof(stockArray[i]) == "object"){
			Object.keys(stockArray[i]).forEach(function(stock){
				var nota = 0.0;

				var patrLiq = parseFloat(stockArray[i][stock]["Pat.Liq"].replace(/\./g, '').replace(/\,/g, '.'));
				if(comparadores.patrLiq.checked &&  patrLiq > parseFloat(comparadores.patrLiq.value))
				    nota = nota + 1
				var liqCorr = parseFloat(stockArray[i][stock]["Liq.Corr."].replace(/\./g, '').replace(/,/g, '.'));
				if(comparadores.liqCorr.checked &&  liqCorr > parseFloat(comparadores.liqCorr.value))
				    nota = nota + 1
				var roe = parseFloat(stockArray[i][stock]["ROE"].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.roe.checked &&  roe > parseFloat(comparadores.roe.value)) 
				    nota = nota + 1
				var divPat = parseFloat(stockArray[i][stock]["Div.Brut/Pat."].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.divPat.checked &&  divPat * 100 < parseFloat(comparadores.divPat.value) && divPat > 0) 
				    nota = nota + 1
				var cresc = parseFloat(stockArray[i][stock]["Cresc.5a"].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.cresc.checked &&  cresc > parseFloat(comparadores.cresc.value)) 
				    nota = nota + 1
				var pvp = parseFloat(stockArray[i][stock]["P/VP"].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.pvp.checked &&  pvp < parseFloat(comparadores.pvp.value) && pvp > 0) 
				    nota = nota + 1
				var pl = parseFloat(stockArray[i][stock]["P/L"].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.pl.checked &&  pl < parseFloat(comparadores.pl.value) && pl > 0) 
				    nota = nota + 1
				var dy = parseFloat(stockArray[i][stock]["DY"].replace(/\./g, '').replace(/\,/g, '.').replace(/%/g, ''));
				if(comparadores.dy.checked &&  dy > parseFloat(comparadores.dy.value)) 
				    nota = nota + 1
				if(comparadores.plxpvp.checked &&  pl * pvp < parseFloat(comparadores.plxpvp.value))
					nota = nota + 1;

				stockArray[i][stock]["nota"] = (parseFloat(nota) / dividePor * 10.0).toFixed(2);

				// console.log([stock, stockArray[i][stock]["nota"], patrLiq, liqCorr, roe, divPat, cresc, pvp, pl, dy])

			});
		}
	}

	return stockArray;
}



// Clears the table
var resetTable = function() {
	if ( $.fn.DataTable.isDataTable('#resultado') ) {
	  $('#resultado').DataTable().destroy();
	}
	console.log("reseta tabela", $(".table-body").html())
}


// *******************************
// 			View state functions
// *******************************
var hideLoading = function(){
	$('#loading-whell').addClass("hidden");
	$('#info').removeClass("hidden");
}


function initInputs(){
	console.log("chamei");
	$(function() {
		$('#patrLiqInput').val(comparadores.patrLiq.value);
		$('#liqCorrInput').val(comparadores.liqCorr.value);
		$('#divPatInput').val(comparadores.divPat.value);
		$('#dyInput').val(comparadores.dy.value);
		$('#crescInput').val(comparadores.cresc.value);
		$('#roeInput').val(comparadores.roe.value);
		$('#pvpInput').val(comparadores.pvp.value);
		$('#plInput').val(comparadores.pl.value);
		$('#plxpvpInput').val(comparadores.plxpvp.value);

		resultadoClone = $(".table-body").clone();
	})
}

// Save rules
$(document).on('click', '#saveRules', function(){

	// Get if checkbox is checked
	comparadores.patrLiq.checked = $('#patrLiqCheckbox:checked').length;
	comparadores.liqCorr.checked = $('#liqCorrCheckbox:checked').length;
	comparadores.roe.checked = $('#roeCheckbox:checked').length;
	comparadores.divPat.checked = $('#divPatCheckbox:checked').length;
	comparadores.cresc.checked = $('#crescCheckbox:checked').length;
	comparadores.pvp.checked = $('#pvpCheckbox:checked').length;
	comparadores.pl.checked = $('#plCheckbox:checked').length;
	comparadores.dy.checked = $('#dyCheckbox:checked').length;
	comparadores.plxpvp.checked = $('#plxpvpCheckbox:checked').length;

	// get new values
	comparadores.patrLiq.value = $('#patrLiqInput').val();
	comparadores.liqCorr.value = $('#liqCorrInput').val();
	comparadores.roe.value = $('#roeInput').val();
	comparadores.divPat.value = $('#divPatInput').val();
	comparadores.cresc.value = $('#crescInput').val();
	comparadores.pvp.value = $('#pvpInput').val();
	comparadores.pl.value = $('#plInput').val();
	comparadores.dy.value = $('#dyInput').val();
	comparadores.plxpvp.value = $('#plxpvpInput').val();

	resetTable();
	buildTable(arrayStocksHistory[0]);

})

================================================
FILE: client/styles.css
================================================
@keyframes App-logo-spin {
  from { transform: rotate(0deg); }
  to { transform: rotate(360deg); }
}

th {
  background: white;
  position: sticky;
  top: 0;
  z-index: 10;
  background: #dddddd;
  cursor: pointer;
}

.hidden{display:none;}

.screen {
  position: relative;
  width: 100%;
  height: 200px;
}


.loading-wrapper {
  position: relative;
}

.loading-wrapper .loading-screen {
    position: absolute;
    top: 0;
    left: 0;
    right: 0;
    bottom: 0;
    z-index: 10;
    background-color: rgba(250,250,250,0.7);
}

.loading-wheel {
    z-index: 11;
    position: absolute;
    top: 50%;
    left: 50%;
    margin-top: -50px;
    margin-left: -50px;
    border: 8px solid #e3e3e3;
    border-radius: 50%;
    border-top: 8px solid #4e8df9;
    width: 100px;
    height: 100px;
    transform: translateY(-50%);
    -webkit-animation: App-logo-spin 1.5s linear infinite;
    animation: App-logo-spin 1.5s linear infinite;
}

.stock {
  color: #0062ff;
}

.green-line {
  background-color: #cde0cd;
}

================================================
FILE: docker-compose.yml
================================================
version: "3.4"
services:
  mongo:
    network_mode: host
    image: willguitaradmfar/mongo-auth
    environment:
      CREATE_DATABASES: "myapp"
      MONGODB_USER: "mongo_user"
      MONGODB_PASS: "admin123"
    volumes:
      - /var/lib/mongodb:/data/db
    restart: always
    ports:
      - 27017:27017

================================================
FILE: functions/.gitignore
================================================
## Compiled JavaScript files
**/*.js
**/*.js.map

# Typescript v1 declaration files
typings/

node_modules/

================================================
FILE: functions/package.json
================================================
{
  "name": "functions",
  "scripts": {
    "lint": "tslint --project tsconfig.json",
    "build": "tsc",
    "serve": "npm run build && firebase serve --only functions",
    "shell": "npm run build && firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "8"
  },
  "main": "lib/index.js",
  "dependencies": {
    "firebase-admin": "^8.0.0",
    "firebase-functions": "^3.0.0",
    "mysql": "^2.17.1"
  },
  "devDependencies": {
    "@types/mysql": "2.15.6",
    "tslint": "^5.12.0",
    "typescript": "^3.2.2"
  },
  "private": true
}


================================================
FILE: functions/src/index.ts
================================================
import { Change } from 'firebase-functions';
import { DataSnapshot } from 'firebase-functions/lib/providers/database';
import * as Mysql from 'mysql';

// // Start writing Firebase Functions
// // https://firebase.google.com/docs/functions/typescript
//
// export const helloWorld = functions.https.onRequest((request, response) => {
//  response.send("Hello from Firebase!");
// });

export const saveStockHistory = (change: Change<DataSnapshot>, context = null) => {
  // export const saveStockHistory = functions.database.ref('/stocks')
  //   .onWrite((change: Change<DataSnapshot>, context) => {

  const arrayStocksHistory = [];

  const val = change.after.val;


  	Object.keys(val).forEach((key) => {
      // console.log(snapshot.val()[key])
      var object = JSON.parse(val[key]);
      arrayStocksHistory.push(object)
      // console.log(object);
    })
  
    // Sort by date, the first one will be the newest
    arrayStocksHistory.sort((a,b) => {
      var date1: any = new Date(a.date);
      var date2 : any= new Date(b.date);
      return date2 - date1;
    })



  const conn: Mysql.Connection = Mysql.createConnection({
    host: process.env.DB_ENDPOINT,
    user: process.env.DB_USER,
    password: process.env.DB_PASSWORD,
    database: process.env.DATABASE_NAME,
  });


  query(conn, "show tables", {}).then((res) => {
    console.log(res);

    conn.destroy();
  })
    .catch(err => conn.destroy());


  // Grab the current value of what was written to the Realtime Database.
  console.log('Payload:', change, "Context", context);
  return ['Uppercasing', context, change];
  // const uppercase = original.toUpperCase();
  // You must return a Promise when performing asynchronous tasks inside a Functions such as
  // writing to the Firebase Realtime Database.
  // Setting an "uppercase" sibling in the Realtime Database returns a Promise.
  // return snapshot.ref.parent.child('uppercase').set(uppercase);
  // });
};


async function query<T>(conn: Mysql.Connection, queryString: string, substituionValues: Object): Promise<T> {
  const queryOptions: Mysql.QueryOptions = transformIntoMySQLString(queryString, substituionValues);

  return new Promise<T>((resolve, reject) => {
    conn.query(
      queryOptions,
      (err: Mysql.MysqlError | null, results?: any, fields?: Mysql.FieldInfo[]) => {
        if (err) reject(err);
        else {
          console.info('Salvo objeto com sucesso no DB');
          resolve(results);
        }
      }
    );
  });
}

function transformIntoMySQLString(queryString: string, substituionValues: any): Mysql.QueryOptions {

  // Get all keys (they have a ':' in front of the name)
  const allKeys: string[] = queryString.split(' ')
    .filter((word: string) => word.indexOf(':') == 0);

  const allValuesInOrder: Array<string> = allKeys
    .map((key: string) => substituionValues[key]);

  // In case of error 
  if (allKeys.length != allValuesInOrder.length)
    throw (
      { 'queryString': queryString, subsValues: substituionValues }
    );


  // Overwrite the keys with '?'
  let MysqlQueryString: string = queryString;
  for (const key of allKeys) {
    MysqlQueryString = MysqlQueryString.replace(key, '?');
  }

  return {
    sql: MysqlQueryString,
    values: allValuesInOrder
  };
}


const change: any = null;
// {
//   before:
//     DataSnapshot: {
//       app:
//       FirebaseApp: {
//         firebaseInternals_: [],
//         services_: { },
//         isDeleted_: false,
//         name_: '__admin__',
//         options_: [],
//         INTERNAL: [] },
//   instance: 'https://bovespastockratings.firebaseio.com',
//   _path: '/stocks',
//   _data:
//     {
//       '-LiAHa0uatNefaIDZ250': ' {\n    "0": {\n        "GBIO33": {\n            "Cresc.5a": "0,00%",\n            "DY": "0,00%",\n            "Div.Brut/Pat.": "0,30",\n            "EBITDA": "23,44%",\n            "EV/EBIT": "0,00",\n            "Liq.2m.": "1.360.630,00",\n            "Liq.Corr.": "2,03",\n            "Mrg.Liq.": "7,73%",\n            "P/Ativ.Circ.Liq.": "0,00",\n            "P/Ativo": "0,000",\n            "P/Cap.Giro": "0,00",\n            "P/EBIT": "0,00",\n            "P/L": "0,00",\n            "P/VP": "0,00",\n            "PSR": "0,000",\n            "Pat.Liq": "729.236.000,00",\n            "ROE": "8,70%",\n            "ROIC": "19,69%",\n            "cotacao": "7,25"\n        }\n    },\n    "1": {\n        "STBP3": {\n            "Cresc.5a": "-2,25%",\n            "DY": "0,10%",\n            "Div.Brut/Pat.": "0,16",\n            "EBITDA": "5,32%",\n            "EV/EBIT": "58,74",\n            "Liq.2m.": "6.607.860,00",\n            "Liq.Corr.": "1,23",\n            "Mrg.Liq.": "-0,01%",\n            "P/Ativ.Circ.Liq.": "-2,58",\n            "P/Ativo": "1,024",\n            "P/Cap.Giro": "39,43",\n            "P/EBIT": "59,37",\n            "P/L": "-26.388,80",\n            "P/VP": "2,21",\n            "PSR": "3,161",\n            "Pat.Liq": "1.337.190.000,00",\n            "ROE": "-0,01%",\n            "ROIC": "1,93%",\n            "cotacao": "4,43"\n        }\n    }'
//     }
//   }
// }

saveStockHistory(change);

================================================
FILE: functions/tsconfig.json
================================================
{
  "compilerOptions": {
    "module": "commonjs",
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "outDir": "lib",
    "sourceMap": true,
    "strict": true,
    "target": "es2017"
  },
  "compileOnSave": true,
  "include": [
    "src"
  ]
}


================================================
FILE: functions/tslint.json
================================================
{
  "rules": {
    // -- Strict errors --
    // These lint rules are likely always a good idea.

    // Force function overloads to be declared together. This ensures readers understand APIs.
    "adjacent-overload-signatures": true,

    // Do not allow the subtle/obscure comma operator.
    "ban-comma-operator": true,

    // Do not allow internal modules or namespaces . These are deprecated in favor of ES6 modules.
    "no-namespace": true,

    // Do not allow parameters to be reassigned. To avoid bugs, developers should instead assign new values to new vars.
    "no-parameter-reassignment": true,

    // Force the use of ES6-style imports instead of /// <reference path=> imports.
    "no-reference": true,

    // Do not allow type assertions that do nothing. This is a big warning that the developer may not understand the
    // code currently being edited (they may be incorrectly handling a different type case that does not exist).
    "no-unnecessary-type-assertion": true,

    // Disallow nonsensical label usage.
    "label-position": true,

    // Disallows the (often typo) syntax if (var1 = var2). Replace with if (var2) { var1 = var2 }.
    "no-conditional-assignment": true,

    // Disallows constructors for primitive types (e.g. new Number('123'), though Number('123') is still allowed).
    "no-construct": true,

    // Do not allow super() to be called twice in a constructor.
    "no-duplicate-super": true,

    // Do not allow the same case to appear more than once in a switch block.
    "no-duplicate-switch-case": true,

    // Do not allow a variable to be declared more than once in the same block. Consider function parameters in this
    // rule.
    "no-duplicate-variable": [true, "check-parameters"],

    // Disallows a variable definition in an inner scope from shadowing a variable in an outer scope. Developers should
    // instead use a separate variable name.
    "no-shadowed-variable": true,

    // Empty blocks are almost never needed. Allow the one general exception: empty catch blocks.
    "no-empty": [true, "allow-empty-catch"],

    // Functions must either be handled directly (e.g. with a catch() handler) or returned to another function.
    // This is a major source of errors in Cloud Functions and the team strongly recommends leaving this rule on.
    "no-floating-promises": true,

    // Do not allow any imports for modules that are not in package.json. These will almost certainly fail when
    // deployed.
    "no-implicit-dependencies": true,

    // The 'this' keyword can only be used inside of classes.
    "no-invalid-this": true,

    // Do not allow strings to be thrown because they will not include stack traces. Throw Errors instead.
    "no-string-throw": true,

    // Disallow control flow statements, such as return, continue, break, and throw in finally blocks.
    "no-unsafe-finally": true,

    // Expressions must always return a value. Avoids common errors like const myValue = functionReturningVoid();
    "no-void-expression": [true, "ignore-arrow-function-shorthand"],

    // Disallow duplicate imports in the same file.
    "no-duplicate-imports": true,


    // -- Strong Warnings --
    // These rules should almost never be needed, but may be included due to legacy code.
    // They are left as a warning to avoid frustration with blocked deploys when the developer
    // understand the warning and wants to deploy anyway.

    // Warn when an empty interface is defined. These are generally not useful.
    "no-empty-interface": {"severity": "warning"},

    // Warn when an import will have side effects.
    "no-import-side-effect": {"severity": "warning"},

    // Warn when variables are defined with var. Var has subtle meaning that can lead to bugs. Strongly prefer const for
    // most values and let for values that will change.
    "no-var-keyword": {"severity": "warning"},

    // Prefer === and !== over == and !=. The latter operators support overloads that are often accidental.
    "triple-equals": {"severity": "warning"},

    // Warn when using deprecated APIs.
    "deprecation": {"severity": "warning"},

    // -- Light Warnings --
    // These rules are intended to help developers use better style. Simpler code has fewer bugs. These would be "info"
    // if TSLint supported such a level.

    // prefer for( ... of ... ) to an index loop when the index is only used to fetch an object from an array.
    // (Even better: check out utils like .map if transforming an array!)
    "prefer-for-of": {"severity": "warning"},

    // Warns if function overloads could be unified into a single function with optional or rest parameters.
    "unified-signatures": {"severity": "warning"},

    // Prefer const for values that will not change. This better documents code.
    "prefer-const": {"severity": "warning"},

    // Multi-line object literals and function calls should have a trailing comma. This helps avoid merge conflicts.
    "trailing-comma": {"severity": "warning"}
  },

  "defaultSeverity": "error"
}


================================================
FILE: fundamentus.py
================================================
#!/usr/bin/env python3

import re
import urllib.request
import urllib.parse
import http.cookiejar
import time
import lxml

from lxml.html import fragment_fromstring
from collections import OrderedDict
import json
import ast
import datetime
import os
from pymongo import MongoClient
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail


def get_data(*args, **kwargs):
    url = 'http://www.fundamentus.com.br/resultado.php'
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    opener.addheaders = [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201'),
                         ('Accept', 'text/html, text/plain, text/css, text/sgml, */*;q=0.01')]

    # Aqui estão os parâmetros de busca das ações
    # Estão em branco para que retorne todas as disponíveis
    data = {'pl_min':'',
            'pl_max':'',
            'pvp_min':'',
            'pvp_max' :'',
            'psr_min':'',
            'psr_max':'',
            'divy_min':'',
            'divy_max':'',
            'pativos_min':'',
            'pativos_max':'',
            'pcapgiro_min':'',
            'pcapgiro_max':'',
            'pebit_min':'',
            'pebit_max':'',
            'fgrah_min':'',
            'fgrah_max':'',
            'firma_ebit_min':'',
            'firma_ebit_max':'',
            'margemebit_min':'',
            'margemebit_max':'',
            'margemliq_min':'',
            'margemliq_max':'',
            'liqcorr_min':'',
            'liqcorr_max':'',
            'roic_min':'',
            'roic_max':'',
            'roe_min':'',
            'roe_max':'',
            'liq_min':'',
            'liq_max':'',
            'patrim_min':'',
            'patrim_max':'',
            'divbruta_min':'',
            'divbruta_max':'',
            'tx_cresc_rec_min':'',
            'tx_cresc_rec_max':'',
            'setor':'',
            'negociada':'ON',
            'ordem':'1',
            'x':'28',
            'y':'16'}

    with opener.open(url, urllib.parse.urlencode(data).encode('UTF-8')) as link:
        content = link.read().decode('ISO-8859-1')

    pattern = re.compile('<table id="resultado".*</table>', re.DOTALL)
    reg = re.findall(pattern, content)[0]
    page = fragment_fromstring(reg)
    lista = OrderedDict()

    stocks = page.xpath('tbody')[0].findall("tr")

    for i in range(0, len(stocks)):
        lista[i] = {
            stocks[i].getchildren()[0][0].getchildren()[0].text: {
                'cotacao': stocks[i].getchildren()[1].text,
               'P/L': stocks[i].getchildren()[2].text,
               'P/VP': stocks[i].getchildren()[3].text,
               'PSR': stocks[i].getchildren()[4].text,
               'DY': stocks[i].getchildren()[5].text,
               'P/Ativo': stocks[i].getchildren()[6].text,
               'P/Cap.Giro': stocks[i].getchildren()[7].text,
               'P/EBIT': stocks[i].getchildren()[8].text,
               'P/Ativ.Circ.Liq.': stocks[i].getchildren()[9].text,
               'EV/EBIT': stocks[i].getchildren()[10].text,
               'EBITDA': stocks[i].getchildren()[11].text,
               'Mrg. Ebit': stocks[i].getchildren()[12].text,
               'Mrg.Liq.': stocks[i].getchildren()[13].text,
               'Liq.Corr.': stocks[i].getchildren()[14].text,
               'ROIC': stocks[i].getchildren()[15].text,
               'ROE': stocks[i].getchildren()[16].text,
               'Liq.2m.': stocks[i].getchildren()[17].text,
               'Pat.Liq': stocks[i].getchildren()[18].text,
               'Div.Brut/Pat.': stocks[i].getchildren()[19].text,
               'Cresc.5a': stocks[i].getchildren()[20].text
               }
            }


    return lista


def get_specific_data(stock):
    url = "http://www.fundamentus.com.br/detalhes.php?papel=" + stock
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    opener.addheaders = [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201'),
                         ('Accept', 'text/html, text/plain, text/css, text/sgml, */*;q=0.01')]
    
    # Get data from site
    link = opener.open(url, urllib.parse.urlencode({}).encode('UTF-8'))
    content = link.read().decode('ISO-8859-1')

    # Get all table instances
    pattern = re.compile('<table class="w728">.*</table>', re.DOTALL)
    reg = re.findall(pattern, content)[0]
    reg = "<div>" + reg + "</div>"
    page = fragment_fromstring(reg)
    all_data = {}

    # There is 5 tables with tr, I will get all trs
    all_trs = []
    all_tables = page.xpath("table")

    for i in range(0, len(all_tables)):
        all_trs = all_trs + all_tables[i].findall("tr")

    # Run through all the trs and get the label and the
    # data for each line
    for tr_index in range(0, len(all_trs)):
        tr = all_trs[tr_index]
        # Get into td
        all_tds = tr.getchildren()
        for td_index in range(0, len(all_tds)):
            td = all_tds[td_index]


            label = ""
            data = ""

            # The page has tds with contents and some 
            # other with not
            if (td.get("class").find("label") != -1):
                # We have a label
                for span in td.getchildren():
                    if (span.get("class").find("txt") != -1):
                        label = span.text

                # If we did find a label we have to look 
                # for a value 
                if (label and len(label) > 0):
                    next_td = all_tds[td_index + 1]

                    if (next_td.get("class").find("data") != -1):
                        # We have a data
                        for span in next_td.getchildren():
                            if (span.get("class").find("txt") != -1):
                                if (span.text):
                                    data = span.text
                                else:
                                    # If it is a link
                                    span_children = span.getchildren()
                                    if (span_children and len(span_children) > 0):
                                        data = span_children[0].text

                                # Include into dict
                                all_data[label] = data

                                # Erase it
                                label = ""
                                data = ""

    return all_data



def send_email(message):
    try:
        sg = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))
        response = sg.send(message)
        print(response.status_code)
    except Exception as e:
        print("Deu erro na hora de enviar o email", str(e))
    


try: 
    if __name__ == '__main__':
        from waitingbar import WaitingBar
        
        THE_BAR = WaitingBar('[*] Downloading...')

        lista = get_data()
        THE_BAR.stop()
        # print("Get all stocks data")


        #Transform em uma lista, agora preciso passar para formato JSON
        array_format = list(lista.items())

        # print(array_format, len(array_format))

        hashes_list = []
        # First include the list of all hashes
        for i in range(0, len(array_format)):
            hashes_list.append(array_format[i][1])

        stocks = []
        # Then from a list of hashes we will transform to a list of stocks
        for i in range(0, len(hashes_list)):
            for key in hashes_list[i]:
                # Adds stockCode
                hashes_list[i][key]["stockCode"] = key
                stocks.append(hashes_list[i][key])
        


    #     json_format = {
    #     "0": {
    #         "DAGB33": {
    #             "Cresc.5a": "46,43%",
    #             "DY": "0,00%",
    #             "Div.Brut/Pat.": "1,37",
    #             "EBITDA": "4,75%",
    #             "EV/EBIT": "0,00",
    #             "Liq.2m.": "916.730,00",
    #             "Liq.Corr.": "1,16",
    #             "Mrg.Liq.": "0,38%",
    #             "P/Ativ.Circ.Liq.": "0,00",
    #             "P/Ativo": "0,000",
    #             "P/Cap.Giro": "0,00",
    #             "P/EBIT": "0,00",
    #             "P/L": "0,00",
    #             "P/VP": "0,00",
    #             "PSR": "0,000",
    #             "Pat.Liq": "9.803.230.000,00",
    #             "ROE": "-0,47%",
    #             "ROIC": "4,59%",
    #             "cotacao": "480,00",
    #             "nota": 0.5
    #         }
    #     }
    # }

        # Calculate the score of the stock
        final_stocks = []
        for stock in stocks:
            nota = 0
            patrLiq = float(stock["Pat.Liq"].replace('.', '').replace(',', '.'))
            if patrLiq > 2000000000:
                nota = nota + 1
            liqCorr = float(stock["Liq.Corr."].replace('.', '').replace(',', '.').replace('%', ''))
            if liqCorr > 1.5:
                nota = nota + 1
            roe = float(stock["ROE"].replace('.', '').replace(',', '.').replace('%', ''))
            if roe > 20: 
                nota = nota + 1
            divPat = float(stock["Div.Brut/Pat."].replace('.', '').replace(',', '.').replace('%', ''))
            if divPat < 0.5 and divPat > 0: 
                nota = nota + 1
            cresc = float(stock["Cresc.5a"].replace('.', '').replace(',', '.').replace('%', ''))
            if cresc > 5: 
                nota = nota + 1
            pvp = float(stock["P/VP"].replace('.', '').replace(',', '.').replace('%', ''))
            if pvp < 2 and pvp > 0: 
                nota = nota + 1
            pl = float(stock["P/L"].replace('.', '').replace(',', '.').replace('%', ''))
            if pl < 15 and pl > 0: 
                nota = nota + 1
            dy = float(stock["DY"].replace('.', '').replace(',', '.').replace('%', ''))
            if dy > 2.5: 
                nota = nota + 1
            stock["nota"] = float(nota) / 8.0 * 10.0

            newStock = {}
            newStock["stockCode"] = stock["stockCode"]
            newStock["patrimonioLiquido"] = patrLiq
            newStock["liquidezCorrente"] = liqCorr
            newStock["ROE"] = roe
            newStock["divSobrePatrimonio"] = divPat
            newStock["crescimentoCincoAnos"] = cresc
            newStock["precoSobreVP"] = pvp
            newStock["precoSobreLucro"] = pl
            newStock["dividendos"] = dy
            newStock["score"] = stock["nota"]
            newStock["stockPrice"] = float(stock["cotacao"].replace('.', '').replace(',', '.'))
            newStock["PSR"] = float(stock["PSR"].replace('.', '').replace(',', '.'))
            newStock["precoSobreAtivo"] = float(stock['P/Ativo'].replace('.', '').replace(',', '.'))
            newStock["precoSobreCapitalGiro"] = float(stock['P/Cap.Giro'].replace('.', '').replace(',', '.'))
            newStock["precoSobreEBIT"] = float(stock['P/EBIT'].replace('.', '').replace(',', '.'))
            newStock["precoSobreAtivoCirculante"] = float(stock['P/Ativ.Circ.Liq.'].replace('.', '').replace(',', '.'))
            newStock["EVSobreEBIT"] = float(stock['EV/EBIT'].replace('.', '').replace(',', '.'))
            newStock["margemEBIT"] = float(stock["EBITDA"].replace('.', '').replace(',', '.').replace('%', ''))
            newStock["margemLiquida"] = float(stock['Mrg.Liq.'].replace('.', '').replace(',', '.').replace('%', ''))
            newStock["ROIC"] = float(stock["ROIC"].replace('.', '').replace(',', '.').replace('%', ''))
            newStock["liquidezDoisMeses"] = float(stock['Liq.2m.'].replace('.', '').replace(',', '.').replace('%', ''))
            newStock["timestamp"] = datetime.datetime.now()

            # Get more information
            print("Getting more information from stock ", newStock["stockCode"])
            specific_data = get_specific_data(newStock["stockCode"])


            # Add everything to the object
            newStock["tipo"] = specific_data['Tipo']
            newStock["name"] = specific_data['Empresa']
            newStock["setor"] = specific_data['Setor']
            newStock["subsetor"] = specific_data['Subsetor']
            newStock["max52sem"] = float(specific_data['Max 52 sem'].replace('.', '').replace(',', '.')) if 'Max 52 sem' in specific_data and not "-" in specific_data['Max 52 sem'] else 0
            newStock["volMed2M"] = float(specific_data['Vol $ méd (2m)'].replace('.', '').replace(',', '.')) if 'Vol $ méd (2m)' in specific_data and not "-" in specific_data['Vol $ méd (2m)'] else 0
            newStock["valorMercado"] = float(specific_data['Valor de mercado'].replace('.', '').replace(',', '.')) if 'Valor de mercado' in specific_data and not "-" in specific_data['Valor de mercado'] else 0
            newStock["valorFirma"] = float(specific_data['Valor da firma'].replace('.', '').replace(',', '.')) if 'Valor da firma' in specific_data and not "-" in specific_data['Valor da firma'] else 0
            newStock["nAcoes"] = float(specific_data['Nro. Ações'].replace('.', '').replace(',', '.')) if 'Nro. Ações' in specific_data and not "-" in specific_data['Nro. Ações'] else 0
            newStock["lucroPorAcao"] = float(specific_data['LPA'].replace('.', '').replace(',', '.')) if 'LPA' in specific_data and not "-" in specific_data['LPA'] else 0
            newStock["margemBruta"] = float(specific_data['Marg. Bruta'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Marg. Bruta' in specific_data and not "-" in specific_data['Marg. Bruta'] else 0
            newStock["EBITsobreAtivo"] = float(specific_data['EBIT / Ativo'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'EBIT / Ativo' in specific_data and not "-" in specific_data['EBIT / Ativo'] else 0
            newStock["giroAtivos"] = float(specific_data['Giro Ativos'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Giro Ativos' in specific_data and not "-" in specific_data['Giro Ativos'] else 0
            newStock["ativo"] = float(specific_data['Ativo'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Ativo' in specific_data and not "-" in specific_data['Ativo'] else 0
            newStock["divBruta"] = float(specific_data['Dív. Bruta'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Dív. Bruta' in specific_data and not "-" in specific_data['Dív. Bruta'] else 0
            newStock["divLiquida"] = float(specific_data['Dív. Líquida'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Dív. Líquida' in specific_data and not "-" in specific_data['Dív. Líquida'] else 0
            newStock["disponibilidades"] = float(specific_data['Disponibilidades'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Disponibilidades' in specific_data and not "-" in specific_data['Disponibilidades'] else 0
            newStock["receitaLiquida"] = float(specific_data['Receita Líquida'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Receita Líquida' in specific_data and not "-" in specific_data['Receita Líquida'] else 0
            newStock["lucroLiquido"] = float(specific_data['Lucro Líquido'].replace('.', '').replace(',', '.').replace('%', '').replace("\n", "")) if 'Lucro Líquido' in specific_data and not "-" in specific_data['Lucro Líquido'] else 0

            
            final_stocks.append(newStock)



        # Saves in mongoDB
        client = MongoClient(os.environ['MONGO_URI'])

        print("Peguei todos os stocks, agora preciso salvar")


        recent_stocks_db = client.recentStocks
        all_stocks_coll = recent_stocks_db.stocks

        # First drop the collection
        recent_stocks_db.stocks.drop()

        for i in range(0, len(final_stocks)):
            stock = final_stocks[i]
            # Insert in mongo
            stock_id = all_stocks_coll.insert_one(stock).inserted_id
            print("Inserted object on recent Objects ", i + 1, " of ", len(final_stocks), " :",   stock_id)


        # Send email of success
        message = Mail(from_email='danilo_aleixo@hotmail.com', to_emails=os.environ.get('EMAIL_TO'), subject='Cron de analise diária, rodou certinho', html_content='Deu certo')
        send_email(message)

except Exception as e:
    print("Deu erro", str(e))
    # Send email of error
    message = Mail(from_email='danilo_aleixo@hotmail.com', to_emails=os.environ.get('EMAIL_TO'), subject='ERROR: Cron de analise diária, deu merda', html_content='Deu errado em ' + str(e))
    send_email(message)





================================================
FILE: gettingstarted/__init__.py
================================================


================================================
FILE: gettingstarted/settings.py
================================================
"""
Django settings for gettingstarted project.

Generated by 'django-admin startproject' using Django 2.0.

For more information on this file, see
https://docs.djangoproject.com/en/2.0/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/2.0/ref/settings/
"""

import os
import django_heroku


# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'CHANGE_ME!!!! (P.S. the SECRET_KEY environment variable will be used, if set, instead).'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'hello'
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'gettingstarted.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'gettingstarted.wsgi.application'


# Database
# https://docs.djangoproject.com/en/2.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}


# Password validation
# https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/2.0/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.0/howto/static-files/

STATIC_URL = '/static/'

django_heroku.settings(locals())

================================================
FILE: gettingstarted/static/humans.txt
================================================


================================================
FILE: gettingstarted/urls.py
================================================
from django.conf.urls import include, url
from django.urls import path

from django.contrib import admin
admin.autodiscover()

import hello.views

# Examples:
# url(r'^$', 'gettingstarted.views.home', name='home'),
# url(r'^blog/', include('blog.urls')),

urlpatterns = [
    url(r'^$', hello.views.index, name='index'),
    url(r'^db', hello.views.db, name='db'),
    path('admin/', admin.site.urls),
]


================================================
FILE: gettingstarted/wsgi.py
================================================
"""
WSGI config for gettingstarted project.

It exposes the WSGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/1.6/howto/deployment/wsgi/
"""

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "gettingstarted.settings")

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()


================================================
FILE: hello/__init__.py
================================================


================================================
FILE: hello/admin.py
================================================
from django.contrib import admin

# Register your models here.


================================================
FILE: hello/migrations/0001_initial.py
================================================
# -*- coding: utf-8 -*-
# Generated by Django 1.9.1 on 2016-01-27 21:54
from __future__ import unicode_literals

from django.db import migrations, models


class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Greeting',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('when', models.DateTimeField(auto_now_add=True, verbose_name=b'date created')),
            ],
        ),
    ]


================================================
FILE: hello/migrations/__init__.py
================================================


================================================
FILE: hello/models.py
================================================
from django.db import models

# Create your models here.
class Greeting(models.Model):
    when = models.DateTimeField('date created', auto_now_add=True)


================================================
FILE: hello/templates/base.html
================================================
<!DOCTYPE html>
<html>
<head>
<title>Python Getting Started on Heroku</title>
  <link rel="stylesheet" type="text/css" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css" />
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js"></script>
  <script type="text/javascript" src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
  <style type="text/css">
    .jumbotron {
        background: #532f8c;
        color: white;
        padding-bottom: 80px
    }
    .jumbotron .btn-primary {
        background: #845ac7;
        border-color: #845ac7
    }
    .jumbotron .btn-primary:hover {
        background: #7646c1
    }
    .jumbotron p {
        color: #d9ccee;
        max-width: 75%;
        margin: 1em auto 2em
    }
    .navbar+.jumbotron {
        margin-top: -20px
    }
    .jumbotron .lang-logo {
        display: block;
        background: #b01302;
        border-radius: 50%;
        overflow: hidden;
        width: 100px;
        height: 100px;
        margin: auto;
        border: 2px solid white
    }
    .jumbotron .lang-logo img {
        max-width: 100%
    }


  </style>
</head>
<body>
<nav class="navbar navbar-default navbar-static-top navbar-inverse">
  <div class="container">
    <ul class="nav navbar-nav">
      <li class="active">
        <a href="/"><span class="glyphicon glyphicon-home"></span> Home</a>
      </li>
      <li>
        <a href="https://devcenter.heroku.com/articles/how-heroku-works"><span class="glyphicon glyphicon-user"></span> How Heroku Works</a>
      </li>
      <li class="dropdown">
        <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false"><span class="glyphicon glyphicon-info-sign"></span> Getting Started Guides <span class="caret"></span></a>
          <ul class="dropdown-menu" role="menu">
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-ruby">Getting Started with Ruby on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-nodejs">Getting Started with Node on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-php">Getting Started with PHP on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-python">Getting Started with Python on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-java">Getting Started with Java on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-clojure">Getting Started with Clojure on Heroku</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-scala">Getting Started with Scala on Heroku</a></li>
            <li class="divider"></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-heroku-and-connect-without-local-dev">Getting Started on Heroku with Heroku Connect</a></li>
            <li><a href="https://devcenter.heroku.com/articles/getting-started-with-jruby">Getting Started with Ruby on Heroku (Microsoft Windows)</a></li>
          </ul>
      </li>
    </ul>
    <ul class="nav navbar-nav navbar-right">
      <li class="navbar-right">
        <a href="https://devcenter.heroku.com"><span class="glyphicon glyphicon-book"></span> Heroku Dev Center</a>
      </li>
    </ul>
  </div>
</nav>

{% block content %}{% endblock %}

</body>
</html>


================================================
FILE: hello/templates/db.html
================================================
{% extends "base.html" %}
{% load staticfiles %}

{% block content %}
  <div class="container">


<h2>Page View Report</h2>


<ul>

{% for greeting in greetings %}
    <li>{{ greeting.when }}</li>
{% endfor %}

</ul>

</div>

{% endblock %}

================================================
FILE: hello/templates/index.html
================================================
{% extends "base.html" %}
{% load staticfiles %}

{% block content %}

<div class="jumbotron text-center">
  <div class="container">
    <a href="/" class="lang-logo">
      <img src="{% static 'lang-logo.png'%}">
    </a>
    <h1>Getting Started with Python on Heroku</h1>
    <p>This is a sample Python application deployed to Heroku. It's a reasonably simple app - but a good foundation for understanding how to get the most out of the Heroku platform.</p>
    <a type="button" class="btn btn-lg btn-default" href="https://devcenter.heroku.com/articles/getting-started-with-python"><span class="glyphicon glyphicon-flash"></span> Getting Started with Python</a>
    <a type="button" class="btn btn-lg btn-primary" href="https://github.com/heroku/python-getting-started"><span class="glyphicon glyphicon-download"></span> Source on GitHub</a>
  </div>
</div>
<div class="container">
  <div class="alert alert-info text-center" role="alert">
    To deploy your own copy, and learn the fundamentals of the Heroku platform, head over to the <a href="https://devcenter.heroku.com/articles/getting-started-with-python" class="alert-link">Getting Started with Python on Heroku</a> tutorial.
  </div>
  <hr>
  <div class="row">
    <div class="col-md-6">
      <h3><span class="glyphicon glyphicon-info-sign"></span> How this sample app works</h3>
      <ul>
        <li>This app was deployed to Heroku, either using Git or by using <a href="https://github.com/heroku/python-getting-started">Heroku Button</a> on the repository.</li>

        <li>When Heroku received the source code, it fetched all the dependencies in the <a href="https://github.com/heroku/python-getting-started/blob/master/Pipfile">Pipfile</a>, creating a deployable slug.</li>
        <li>The platform then spins up a dyno, a lightweight container that provides an isolated environment in which the slug can be mounted and executed.</li>
        <li>You can scale your app, manage it, and deploy over <a href="https://addons.heroku.com/">150 add-on services</a>, from the Dashboard or CLI.</li>
      </ul>
    </div>
    <div class="col-md-6">
      <h3><span class="glyphicon glyphicon-link"></span> Next Steps</h3>
      <ul>
        <li>If you are following the <a href="https://devcenter.heroku.com/articles/getting-started-with-python">Getting Started</a> guide, then please head back to the tutorial and follow the next steps!</li>
        <li>If you deployed this app by deploying the Heroku Button, then in a command line shell, run:</li>
        <ul>
          <li><code>git clone https://github.com/heroku/python-getting-started.git</code> - this will create a local copy of the source code for the app</li>
          <li><code>cd python-getting-started</code> - change directory into the local source code repository</li>
          <li><code>heroku git:remote -a &lt;your-app-name></code> - associate the Heroku app with the repository</li>
          <li>You'll now be set up to run the app locally, or <a href="https://devcenter.heroku.com/articles/getting-started-with-python#push-local-changes">deploy changes</a> to Heroku</li>
        </ul>
      </ul>
      <h3><span class="glyphicon glyphicon-link"></span> Helpful Links</h3>
      <ul>
        <li><a href="https://www.heroku.com/home">Heroku</a></li>
        <li><a href="https://devcenter.heroku.com/">Heroku Dev Center</a></li>
        <li><a href="https://devcenter.heroku.com/articles/getting-started-with-python">Getting Started with Python on Heroku</a></li>
        <li><a href="https://devcenter.heroku.com/articles/django-app-configuration">Configuring Django Apps for Heroku</a></li>
      </ul>
    </div>
  </div> <!-- row -->
   <div class="alert alert-info text-center" role="alert">
    Please do work through the Getting Started guide, even if you do know how to build such an application.  The guide covers the basics of working with Heroku, and will familiarize you with all the concepts you need in order to build and deploy your own apps.
  </div>
</div>
{% endblock %}


================================================
FILE: hello/tests.py
================================================
from django.contrib.auth.models import AnonymousUser, User
from django.test import TestCase, RequestFactory

from .views import index

class SimpleTest(TestCase):
    def setUp(self):
        # Every test needs access to the request factory.
        self.factory = RequestFactory()

    def test_details(self):
        # Create an instance of a GET request.
        request = self.factory.get('/')
        request.user = AnonymousUser()

        # Test my_view() as if it were deployed at /customer/details
        response = index(request)
        self.assertEqual(response.status_code, 200)

================================================
FILE: hello/views.py
================================================
from django.shortcuts import render
from django.http import HttpResponse

from .models import Greeting

# Create your views here.
def index(request):
    # return HttpResponse('Hello from Python!')
    return render(request, 'index.html')


def db(request):

    greeting = Greeting()
    greeting.save()

    greetings = Greeting.objects.all()

    return render(request, 'db.html', {'greetings': greetings})



================================================
FILE: index.html
================================================
<!DOCTYPE html>
<html class="no-js" lang="en">



<head>
		<meta http-equiv="refresh" content="0; url=https://bovespastockratings.firebaseapp.com" />
</head>
</html>


================================================
FILE: manage.py
================================================
#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "gettingstarted.settings")

    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)


================================================
FILE: requirements.txt
================================================
asn1crypto==0.24.0
certifi==2019.9.11
cffi==1.12.3
chardet==3.0.4
crypto==1.4.1
cryptography==3.3.2
dnspython==1.16.0
gcloud==0.18.3
googleapis-common-protos==1.6.0
httplib2==0.19.0
idna==2.8
jwcrypto==0.6.0
lxml==4.6.3
Naked==0.1.31
oauth2client==4.1.3
protobuf==3.10.0rc1
pyasn1==0.4.7
pyasn1-modules==0.2.6
pycparser==2.19
pycryptodome==3.6.6
pymongo==3.9.0
python-jwt==3.2.4
PyYAML==5.4
requests==2.22.0
requests-toolbelt==0.9.1
rsa==4.7
shellescape==3.4.1
six==1.12.0
sseclient==0.0.24
urllib3==1.26.5
sendgrid==6.1.1


================================================
FILE: services.sh
================================================
#!/bin/bash
docker-compose -f docker-compose.yml up -d

================================================
FILE: waitingbar.py
================================================
#!/usr/bin/env python3

import sys
import threading
import time

from itertools import cycle


class WaitingBar(object):
    '''
    This class prints a fancy waiting bar with Greek chars and spins.
    It uses a thread to keep printing the bar while the main program runs

    Usage:

    THE_BAR = WaitingBar('Your Message Here')
    # Do something slow here
    (...)
    THE_BAR.stop()
    
    copyright phoemur - 2016
    '''

    def __init__(self, message='[*] Wait until loading is complete...'):
        self.MESSAGE = ' ' + str(message)
        self.CYCLES = ['-', '-', '\\', '\\', '|', '|', '/', '/', '-', '-', '\\', '\\', '|', '|', '/', '/']
        self.intab = u'abcdefghijklmnopqrstuvwxyzáàãâéèẽêíìîĩóòôõúùũûçABCDEFGHIJKLMNOPQRSTUVWXYZÁÀÃÂÉÈẼÊÍÌÎĨÓÒÔÕÚÙŨÛÇ'
        self.outab = u'αβ¢ΔεϝγηιφκλμνΩπσϼΣτυϞωχψζααααεεεειιιιΩΩΩΩυυυυ¢αβ¢ΔεϝγηιφκλμνΩπσϼΣτυϞωχψζααααεεεειιιιΩΩΩΩυυυυ¢'
        self.TABLE = {x: y for x, y in zip(self.intab, self.outab)}

        self.event = threading.Event()
        self.waiting_bar = threading.Thread(target=self.start, args=(self.event,))
        self.waiting_bar.start()

    def start(self, e):
        for index in cycle(range(len(self.MESSAGE))):
            if e.is_set():
                break
            if not self.MESSAGE[index].isalpha():
                continue
            for c in self.CYCLES:
                buff = list(self.MESSAGE)
                buff.append(c)

                try:
                    if sys.stdout.encoding.upper() == 'UTF-8':
                        buff[index] = self.TABLE[buff[index]]
                    else:
                        buff[index] = buff[index].swapcase()
                except KeyError:
                    pass

                sys.stdout.write(''.join(buff))
                time.sleep(0.05)
                sys.stdout.write('\r')
                sys.stdout.flush()

    def stop(self):
        self.event.set()
        self.waiting_bar.join()
        sys.stdout.write(self.MESSAGE + ' \n')


if __name__ == '__main__':
    '''
    A simple example to demonstrate the class in action
    '''

    # Start the bar
    THE_BAR = WaitingBar('[*] Calculating useless stuff...')

    # Do something slow
    import math
    from pprint import pprint

    a_list = {a: b for a, b in zip(range(1, 41), map(math.factorial, range(1, 41)))}
    time.sleep(20)

    # Stop the bar and print result
    THE_BAR.stop()
    pprint(a_list)