So I got tired of having 4 or 5 different scraping tools eating up my resources. I don’t live in the middle of the city. I’m just 5 minutes outside of the edge of the universe. So my internet speeds are a bit slower and I’m just running one machine most of the time. So when I started doing some heavier scraping, and automating web 2.0 construction, plus my regular testing of new products and services. I quickly find that I’m bored because of bandwidth and computer issues.
What is the Solution to my Resources Problem?
I just created a few VPS, one for each of my main scraping programs. I’m actually still running scrapebox locally just because the serial is already associated with this machine. Soon I’ll move it to its own VPS. You’re probably thinking, aren’t VPS expensive? Well they’re pretty reasonable, actually but this system I’m using currently is 100% free for the first year.
Yes. I’ve got a few machines on some of the best servers in the world, with some of the fastest connections. They’re on amazon servers, and just running VPS isn’t all you can do with these machines. You can host websites, or your own app. Much more capabilities that I’m able to explain in a single blog post. I will go over how I set up my VPS. You can follow along with the video if you have trouble.
Once you’ve signed up for AWS, over at amazon, it just takes a short time to get your ‘instance’ set up and running. In the above video you’ll see that it takes less than 10 minutes if you don’t count the 5 minute wait time. I actually messed up initially and still took less than 10 minutes.
Choose to set up an EC2, one instance for a year (the amount of hours that add up to 1 year), is free. I set up a windows machine using Microsoft 2012 R base. It looks like windows 8 and is easy to work with. You may wish to increase the space on the machine. I upgraded to 60gb without issues, when I had set up with 100gb of space it gave me a warning that this setup didn’t meet the requirements for a free setup.
I then launch the instance and I am able to set up the key for getting access to the virtual machine. Once downloaded I used this encrypted key to find out the login password for my completed installation. The installation takes about 5 minutes. Then I download the remote connection app. I put these on my desktop and use the program I plan on running in each one as their name.
I can then open the instance on my machine, use the password I decrypted with the key, and begin loading my programs.