Written By Christian Soltis
April 10, 2019
My background is in physics and electrical engineering, so it comes as no surprise that I was pretty stoked about the news of a picture of an actual black hole. I'll leave most of the scientific discussion to the professionals, and instead use this as an opportunity to kick off a series of tutorials about physics simulations. We'll start simple by creating a python program that asks the user for a mass and returns some information like how large a black hole of that mass would be, among other things. As before, the finished file can be found on my github page here.
This tutorial will be a short introduction into doing physics with python. It is all very intuitive, and should be accessible to anyone regardless of their level of physics or math background. We will be using the Schwarzschild radius1 as a measurement of the size of our theoretical black hole. That radius represents the start of the point of no return around a black hole aka, the event horizon. There are two constants that we need to do our math. The first is the gravitational constant. Denoted as
G, it is probably most known from its use in Newton's Universal Law of Gravitation2. The other constant we need is, of course, the speed of light3. It is denoted as
Almost ready to start writing our code. The last thing we need to talk about is exactly what we want our program to do. We could just have it return the radius of a black hole that has the mass given by the user, but why not add in a few more bits of functionality? Something I think is worth including is the radius of the photon sphere. The photon sphere is the spherical area in which gravity is so strong, light must travel in an orbit. What this represents for a black hole is the closest possible stable orbit for anything. When looking at the accretion disk of matter around a black hole like in the picture above, the closest that gets to the event horizon, is three times the radius of the event horizon. Because photons have no mass, they can orbit closer, at one and a half times the radius of the event horizon.
The other thing I think we should include is a calculation of the escape velocity4 at the Schwarzschild radius. This is because of the fact that another definition of the Schwarzschild radius is the radius at which a given mass has an escape velocity that is equal to the speed of light. This is why anything inside of the the Schwarzschild radius of a black hole is black. The escape velocity inside of that radius is greater than the speed of light, disallowing anything from escaping. So we can include the formula for escape velocity in our program and feed it our mass and radius to test that everything is working. Provided it does work, that escape velocity should always be the speed of light.
Let's get started. First we
import a couple things we need to do our math and represent it in a readable way. Beneath that, we should define our constants as global variables since we will never want these to change and will want to use them everywhere. Using the values listed above, assign values to
Now let's define a function that asks the user for a mass value, finds the Schwarzschild radius of a black hole with that mass, finds the radius of the photon sphere for that black hole, and finds the escape velocity at that Schwarzschild radius. This is all pretty straightforward stuff, just putting those equations into a form our program can use. However, something we haven't done before is us the Decimal library. In this program we are using it to keep our values at a reasonable length with just two decimals. To do this, we can make three variables that take our
float values from the equations and save them as strings with two decimals. Next, add three
if __name__ == '__main__': statement below our function that calls it.
All finished. If you want to check your program to see that it is printing the correct values, look at this Wikipedia page. It has a table of things like the sun, a human, and even a big mac with their corresponding Schwarzschild radii. This is meant to be the first in a series, so hopefully the next tutorial is out soon. I'd like to get into doing animations that demonstrate physics phenomena. That is likely to be where this series heads. In the meantime, feel free to contact me via the contact page with any suggestions for tutorials.
Written By Christian Soltis
April 2, 2019
Financial computing has a relatively high barrier to entry compared to other technical topics. With machine learning for instance, there are a million tutorials and resources out there to help you get started. That is not quite the case for finance related topics in my experience. This tutorial on how to create an options pricing program will hopefully put a dent in that problem. The finished file can be found here if you want to skip the tutorial.
The first thing you'll notice if you've read anything else on this website is that this tutorial is not in python. While it totally could be, I figured since efficiency is the name of the game in financial computing that I could use this as an opportunity to introduce some c++ to the site. Something to keep in mind while reading this tutorial is that understanding the math and finance behind how this program works is not necessary to start messing around with options pricing, but it can help. I'll cover the very basic aspects of these things, but if you get confused or want to learn more, I recommend looking for some other sources that go into more detail. I will likely cover the math more specifically in a later tutorial.
A few short things to cover before we start writing code. To oversimplify a little, an option is an agreement that allows you to either buy or sell a given security at a certain price before or on an agreed upon date. If you expect the price of that security to rise, that is a "call" option. If you expect the price to go down, that's a "put". The math to price both of these is very similar, and both will be covered by our program.
That's the Black-Scholes formula. It is one of the ways options are priced. Without going into too much detail, the variables in that formula are (ignoring subscripts as they are unnecessary):
S- stock price
K- the strike price or the price at which the option can be exercised
N()- the cumulative normal distribution function (more about this later)
r- the risk-free interest rate
t- time until option is to be exercised as a percentage of a year
σ- greek letter sigma which represents volatility as a percentage
I left out
d2 because those are more complicated.
d1 is especially so.
d2 is a little easier to understand. It is the risk-adjusted probability that the option is going to be exercised. Regardless of their technical definitions, these two little formulas contain an essential part of the math that accounts for some of the more mercurial aspects of options pricing. Seeing as I am not equipped to adequately explain them, let's just accept them and move on!
Onto the code. I'm using Visual Studio Code as my development environment for this project and g++ as my compiler. If you're using a different setup, your mileage may vary. First we need our
#include lines that give our program the ability to do math and print out to the console. After that we will write
using namespace std; so we don't have to write
std:: before all our standard functions. This is technically bad practice due to namespace pollution, but for a small project like this, it won't cause any problems. Just know that for anything serious, you should never use a namespace like that.
The first function we are going to write is going to handle the
N() term in the formula. This is the cumulative normal distribution function. Knowing the underlying math is out of the scope of this tutorial, but you should at least know that this function will take an input and map it somewhere between 0 and 1. Thanks to
cmath, this function can be boiled down to just a return line with our math in it.
With that function down, the next step is to write the function that does the rest of the math. We will call this
callOption(), and it will take all our variables from above as arguments. Next we declare some floats that will hold the results of the formulas, one for
d1, one for
d2, and one for
price that holds our answer. The hardest part of the whole tutorial is getting the formulas typed into our program correctly. It took me way too long to do this. Luckily you can just copy and paste from the snippet below! Lastly the function sends our answer to the console.
Since we made a function for call options, it makes sense to include one for put options as well. It will be nearly identical to our
callOptions function with just a few minor tweaks to the formulas.
Nearly finished. All we need now is our
main() function that defines our variables and gives them to our other functions. We can declare them all on one line luckily. After that we just have a few helpful
cout lines to make things readable and we call our options functions.
A question you may now have is how to get the values for those variables. I personally use Robinhood to get all the values I need. They support the buying of options on their platform, and they give you all the information you want. This includes the volatility and date of exercise. One thing they won't give you is the risk-free interest rate. According to a friend of mine that works in NYC in the finance industry, the rate most people use for this is the Federal Funds Rate. As of the writing of this tutorial, it is sitting at 2.5%. Note: the interest rate, as well as the volatility percentage, need to be entered in decimal form. So 2.5% should be entered as 0.025. Here is an example, Boeing currently trades at $391.59, and a $395 call option that expires in 16 days costs between $6.90 and $7.05 according to Robinhood. It's implied volatility is 25.63% and the interest rate is still 2.5%. Running that through our program gives us a value of $7.00 per contract. It works!
This was a very surface level introduction to options pricing. I'd like to dive a little deeper at some point. The math behind these calculations is interesting, and we haven't even discussed what "the greeks" are at all. I will definitely write up a more detailed explanation about options in the future. In the meantime, check out some other resources on the topic to get a little more background.
Written By Christian Soltis
March 29, 2019
This is a quick tutorial on how to set up a little Twitter scraper using python and the python-twitter library. Following the finance theme from previous tutorials, we will be looking for stock tickers mentioned in tweets. The bones of this program could easily be used to search for other information as well if you're not interested in grabbing stock tickers. The finished file is located here if you just want to grab the code.
The first step in this project is downloading the python-twitter library. This is done with a short little pip install:
pip3 install python-twitter (Note: it is recommended to do this in a virtual environment). Nice and easy. Next, apply for a developer account on Twitter. They implemented this at the end of last year, so now it takes just a little bit of extra work to get access to their API. You can sign up at their developer site here and follow their mostly adequate instructions. When asked for a website, if you don't have one, you can enter the url to your personal Twitter profile or even your Github page if you have either of those. The other URLs you can just leave blank.
Once you have your developer account set up, navigate to the Apps page, select Create an app, and fill out the information you're asked for. The last thing we need from the developer site right now are the keys and tokens associated with the app we just created. Navigate to the Keys and tokens section at the top of the page. You should see two Consumer API Keys and an option to generate an Access token and access token secret. Generate the token and token secret, and save all four of those keys and tokens. Those are how Twitter will recognize our app and allow us to access Twitter.
Time to start writing our code. First things first, create a
twit_creds.py file where we can store those keys and tokens separately from our main file. Not only does this keep our unique keys and tokens private, it is even required by Twitter in their terms of service should you upload your code somewhere public like Github. Make four variables for your two keys and two tokens and assign your specific info to those variables.
As with all projects, we will start our main python file with imports. All we need for this project is to
import twitter for the library and to grab our credentials from our other file using:
from twit_creds import *. Right beneath that, let's add the code that authenticates our app. I like to use multiple lines to organize this info, but this is technically optional. It just keeps you from having a really long single line.
With the prep work finished, we can now write the two functions we will use to search Twitter. The first function will search and grab tweets according to the presence of a term that we provide.
The code above uses the python-twitter library to grab 100 english language tweets that contain our term and gives them to us in the .json format. The next line just grabs the tweet text out of that. Finally a list of those tweet texts is returned.
All that's left to do is search through the list of tweet texts we have for stock tickers. Create an empty list that will hold our potential stock tickers. We can use a
for loop with an
if statement to grab words that start with "$" and are the right length to be a stock stocker.
.append() those to our empty list so we can see which of those are likely to be stock tickers. Next it makes sense to use list comprehension to make our final list of stock tickers. If you're not familiar with list comprehension, it basically boils down to a list with a defining statement inside the brackets. In this case, we make list called
stock_list and put an if statement that checks the characters of the terms in
possible_stocks after the dollar sign for anything that isn't a letter. This ensures we don't include any dollar amounts in our list of stock tickers.
The last bit is our
if __name__ == '__main__': statement that calls our functions. With that, we are all finished. This code specifically searches for a hardcoded search term. A good idea to progress from here would be to perhaps ask the user which term they'd like to search for. Another possible place to go with this code is to have it perform the search multiple times with different search terms. I encourage you to experiment with different search parameters to fine tune your search results. The documentation for this library can be found here.This can be a powerful tool when performing any sort of trend analysis on the public since it is free to start looking through Twitter this way. Twitter also offers Premium APIs to users who wish to get access to more powerful methods of analysis. I had originally planned to make a tutorial for the C++ twitter library, but ran into difficulties getting it up and running. I will spend more time getting that to work, so I can make that tutorial as well.
Written By Christian Soltis
March 26, 2019
If you've recently decided to make the switch to a Linux OS, there are a few things to familiarize yourself with before you can start to feel truly comfortable with it. Arguably the most important difference between Windows, and to a lesser extent MacOS, and Linux is the use of the terminal. The first time I ever tried Linux, I was a bit put-off by how much you needed to use the terminal. Why couldn't I just click a download link for everything?! I have to just know the name of whatever program I want to install and then type a command in the terminal? Madness! Of course, at this point in my life, I thought a typo in the command line on Windows would all but delete
System32 and brick my PC. Turns out all I needed was to get my feet wet and learn a few handy commands to feel right at home with a Linux OS.
clear. If you've been mucking around in the terminal window, and it's starting to get a little crowded, type
clearby itself and hit enter. Voila, you now have a nice, clean, empty terminal window!
cd. Short for "change directory", this command, followed by the path to your desired directory, moves the focus of your terminal from your
homedirectory to whichever one you want. It is useful to know that
cd ..will move you into the parent folder of wherever you're currently at. You can also use the
tabkey to finish off the directory or filename you're typing, provided you've typed enough of it for the OS to figure it out. The tricky part here is knowing the exact path to your desired directory. Luckily, there's a command that can help!
lscommand is one of the most useful. It is short for "list", and it will print to the terminal a list of the contents of whichever directory you're in. The most frequent usage of this, at least on my computers, is for finding the exact name of a file or folder I'm looking for. For example, if you're attempting to
cdinto a directory you know is several layers deep but can't quite remember the name of that next folder, use the
lscommand to show what's available. Once you know the name of the next folder, type that in after
cdand you're all set.
PROMPT_DIRTRIM=1. That gets rid of all but the current folder name from the terminal. If you want to see more parent folders, just change the number to however many you want to see.
mkdircommand is for you! This command will plop a folder named whatever you type right after it in your current directory. Need a folder for your crummy pixel art that you put on your website?
mkdir CrummyPixelArtis all you need to keep it all in one place.
mvfollowed by the filename and destination to move that file wherever. To continue my previous example, moving the file
badPixelArt.pngto our newly created folder from the last step can be done like so:
mv badPixelArt.png /path/to/CrummyPixelArt.
badPixelArt.pngpicture and want to get rid of it, the
rmcommand will do just that. Short for "remove", it will get rid of files, and empty directories. So to get rid of that awful picture, type
rm badPixelArt.pngwhile in the directory the file is in, and it will be deleted. If you try to remove a folder with files inside of it using
rmyou will get an error. Provided you're sure that there is nothing of value in that folder, you can use the
-rfoption to override that warning. This will look like
rm -rf FolderNameif you are in the directory where that folder is. Otherwise you can type the path to that folder instead of just the folder's name.
vim file.extin the terminal. This turns your terminal window into a text editor. First, press the
ikey to enable editing mode. Now you can use the arrow keys to navigate around the file and change what you need to (yes I know there are more efficient ways to move around, but those aren't necessary for quick edits). Once you're done editing your file, press the
esckey to exit editing mode. If you want to exit and save your work, type
:wq. If you want to exit without saving, type
:q!. If you haven't made any changes and want to exit, you can drop the
!from the previous command. Those are the absolute basics of vim, but there is a massive rabbit hole to go down there. I'll consider making a full vim tutorial in the future.
xkill. This command will turn your cursor into the grim reaper. Whatever window you click after typing that in, will instantly stop running. This is useful for when you forgot an exit statement for your while loop or recursive function, and your computer is starting to freak out. Sometimes good ol'
ctrl + cisn't getting the job done. It is also useful to know that you can press
Alt + F2and enter
xkillthere to achieve the same result.
Written By Christian Soltis
March 24, 2019
My previous tutorial used financial data gathered by a third party service as a focal point. If you're anything like me, you've encountered the difficult task of getting your hands on reliable free financial data. While this won't totally assuage those woes, it can help. I mentioned the application IFTTT (IF This Then That) in the previous tutorial. It worked by putting the closing price of a couple stocks I wanted data on into a Google Spreadsheet everyday. This tutorial will teach you how to get that price data for yourself.
As always, the finished file can be found on my github page if you just want the goods without any fuss. For those of you who like to be told what to do, read on!
Firstly, let's cover what we will need to get the job done. This should be a quick little intro just to get your feet wet, so you won't need much. We will be using the Selenium web scraping package to do our dirty work. The only other thing we will need is the capability of making a virtual environment. Virtual Environments in python are a convenient way of keeping your projects separate from anything else on your machine. The reason you would put something into a virtual environment is to keep all the dependencies working together. If an update to something the project relies on breaks your program, but another project of yours requires that update, then you're stuck between a rock and a hard place.
Step one is to create a directory for our project. Make a folder named
WebScraper and navigate inside it. Now we need to create our virtual environment. To install the module, type
$ sudo apt-get install -y python3-venv into your terminal. Then create the environment directory with
$ python3 -m venv webScraper. Startup your environment using
source $ source webScraper/bin/activate. Lastly, install the Selenium package with
pip install selenium. Optionally, if your directory is taking up too much room in your terminal, you can use the handy command
PROMPT_DIRTRIM=1. This will make your terminal only show the folder you are currently in, giving you more room to work with.
Busy work is finished! This tutorial will simply ask the user for a stock ticker, go to the Yahoo! Finance page for that ticker, grab the current price, and print that out to the console. Easily done in just a dozen or two lines of code.
First few lines of code of course will be our imports. We only need two for this project, but there are many other functionalities in the Selenium package that we just don't need yet. Along with imports and a few lines of housekeeping, we will write an
input statement to get the user's desired stock ticker. It's also a good idea to capitalize that input since that is what the website will be looking for.
Now we can write our short little lookup function that actually scrapes the website. Create a
priceLookup(ticker) function that takes
ticker as an argument. Next we tell the function what browser we want to use, in this case Firefox, and feed that
options=options as an argument to tell it to use our previously defined option when it runs. The option we created earlier tells the browser to run in "headless" mode. Headless mode means that the program doesn't have to actually open up a browser page graphically to do it's work. It can do it all under the hood, theoretically saving us some processing power (you'll see more about this a little later).
The next line we need to write is the one that tells the program which web page we'd like to go to. For this project, I simply went to the Yahoo! Finance page for a stock ticker, and looked at the URL to see where it put my search. In order to go to any stock page the user wants, we just need to cut the ticker out of the URL and concatenate the rest with our
user input. The most difficult part of this whole process is finding something unique about the data you want in the
CSS of the webpage. We need an identifier in order to tell the program where to look. Luckily the Web Console on Firefox helps us out. We are looking for the
CSS Selector in this case as it was the simplest identifier I could find that worked. To find this, navigate to a stock page on Yahoo's website. Press the
F12 key to bring up the console and click the element picker button in the top left of the panel. Then just click the price on the page to highlight the corresponding code in the
HTML shown in the console. Finally, left-click the highlighted code and navigate to Copy > CSS Selector. Now we have our identifier for the current price of any stock on Yahoo! Finance. Give that selector, as a string, as an argument to a
find_element_by_css_selector() method as I've done below. Finish up the function by turning the response into text, closing the browser, and returning the price.
Technically, we are finished! Quick and easy project with no issues whatsoever. Although, it does seem to run pretty slow, at least on my old laptop. We ran the browser in headless mode, so you'd think it would run a little quicker, right? I certainly thought so. It turns out there are some alternatives to the in-house headless mode that Firefox provides. The one I found first is called PhantomJS. This doesn't come without any strings attached, however. PhantomJS support is deprecated by Selenium. They don't plan on continuing to assure that the two will play nicely together. This is why we wrote our function above the way we did. For the time being though, because it offers such an insane jump in speed, let's write another version of our
priceLookup function that uses PhantomJS.
A couple things to highlight about the above code. First, I added another
input statement at the top of the file that asks whether or not the user would like to use PhantomJS. This was mostly used by me to compare the speeds between PhantomJS and Firefox normal headless mode. I encourage you to try them both out as well. The function itself is almost identical with self-explanatory changes. Following the new function, I wrote a few lines that check the answer to that new
input line and print out the responses from the appropriate
And with that, we are actually done! This was a very simple example of how to use Selenium for web scraping, and surely not the last to appear on this website. There are other web scapers for python as well which I would like to compare some day. For now, this should be enough to get you started, especially with finance data since that is hard to come by. It should be noted that, while Yahoo! doesn't expressly forbid web scraping, they aren't likely to be super happy about massive amounts of traffic to their site grabbing data. Be responsible about how you get information from them. Setting up a program that grabs price info on a thousand different stocks every hour is probably a little overkill. Financial data is expensive, so be reasonable about what you grab for free. Hopefully this tutorial was helpful to some of you who wanted to get into web scraping, and look forward a continuation on this tutorial sometime in the future.
Written By Christian Soltis
March 23, 2019
Google sheets is the Google version of Microsoft Excel, and it works wonderfully with python due to the Sheets API Google created. The Sheets API allows you to interact with any spreadsheet you have access to. You can use python to do just about anything you could think of with whatever data you gather, but in this tutorial we will just cover some Matplotlib basics and graph what we grab from the spreadsheet. The completed python file is located here if you would like a reference throughout the tutorial. (Note that this tutorial will assume Python version 3.x).
First things first, installing the necessary tools for the job. There are a few things to grab from Google to get things started with their API. Head to their quickstart guide for python and click the button labeled "Enable The Google Sheets API". Copy down your Client ID and Secret for your own records, and download the Client Configuration .json file.
Next step is to install the Google Client library using their provided
pip install line. Copy it from that page and paste it into your command line or terminal. You'll notice below that line that they have also provided a file called "quickstart.py". We will be using some of this file, but not all of it. It includes code that goes into a spreadsheet they made as a tutorial, but they don't explain it as well as we'd like for a beginner. We will be only using what is actually necessary to access a spreadsheet of our own. The next, and final, install will be Matplotlib. If you have somehow not heard of this library yet, it is a great tool to visualize data, and I have used it many times in college for assignments as well as for personal projects. Depending on your OS, there are several options for the install here. If you're using linux, the install is as easy as
sudo apt-get install python3-matplotlib. Otherwise, you can use pip to install on macOS, Windows, and Linux by using the following :
python -m pip install -U pip python -m pip install -U matplotlib
credentials.jsonfile into this folder and also create a .py file where we will be writing the code that interacts with our spreadsheet. Here is a link to the spreadsheet I'll be using in this tutorial if you don't have one of your own that you'd like to use. It contains stock price data for Tesla and Amazon that an IFTTT* routine put into a Google spreadsheet for me for about a year.
quickstart.pyfile that Google provided in their documentation that we are going to use without all the extraneous stuff.
RANGE_NAMEto the values for your spreadsheet at this time. You can find the spreadsheet ID in the url of your google spreadsheet. It is located between
/editin the URL. The range name for your spreadsheet will be the range of cells you want to look at. For my spreadsheet, I am concerned with columns 'A' through 'E' starting on the second row, so I don't include the column labels in my search. Somewhat confusingly, this would make my range name be 'A2:E' (trying to copy the format of the range name in Google's file doesn't work for whatever reason).
main()function, create the two dictionaries like I have at the top of the code snippet below. At the bottom of the
main()function, we want to write the code that iterates through the spreadsheet so we can grab that data. That is shown at the bottom of the code snippet below.
stockNameas an argument. Next write an if/else statement to check which company, Tesla or Amazon, we want to graph.
date_lstand leave them empty. Write two
for loopsthat iterate through the keys of
appendsthe key to
date_lstand the value to
date_lstright now, and trying to show that all at once would look terrible. A sensible choice here would be to chop up that list into 12 pieces, so that each chunk represents roughly a month. This can be done dynamically by a
for loopthat increments a counting variable, then uses that variable to pick a certain index of the
plt.plot(date_lst, price_lst)tells Matplotlib to plot the data from those lists we made as the x-axis and y-axis respectively.
plt.xticks(x_axis_dates)allows us to override what Matplotlib would normally set as the x-axis intervals, in this case every single date in our list, with a more reasonable 12 dates.
plt.sublots_adjust()allows use to change the proportions and the margins of our graph. I obtained these values, and I encourage you to change them if necessary, by changing them in the graph itself to see the actual numbers. Lastly there is
plt.show(). This simply puts everything together and actually displays the graph.
inputfrom the user. The final step is to activate our program! At the bottom of the file, just underneath our newly written
getInput()function, add an
if __name__ == '__main__':statement that calls our functions.
Written By Christian Soltis
March 19, 2019
I spent an obnoxiously long time trying to configure a cheap WiFi dongle to work on my old Dell laptop that was running Ubuntu. I spent hours scouring the internet in the hopes of finding someone who had the same problem. Luckily that's exactly what happened. Since it took such a large amount of effort to track down the solution to this problem, I've decided to detail it here. This way it's in at least two places on the whole internet...
The following code can be copied and pasted into your terminal. It will get rid of any faulty version of the driver you may have on your system,and replace it with a version that works.
sudo apt purge rtl8812au-dkms sudo apt install git git clone https://github.com/gnab/rtl8812au.git sudo cp -r rtl8812au /usr/src/rtl8812au-4.2.2 sudo dkms add -m rtl8812au -v 4.2.2 sudo dkms build -m rtl8812au -v 4.2.2 sudo dkms install -m rtl8812au -v 4.2.2
Written By Christian Soltis
March 18, 2019
This is a test post to see if I have successfully set up the site on the new host platform. Hopefully this one works a little better...