Abe Massry

Web Development and Building Businesses

Working With Multiple Cores in Python


I just created this new project called Crazip because I wanted to try to write a program using multiple cores in python. Normally I would use threads for something like this because it would cut down on the need for interprocess communication. It turns out that using threads in python doesn’t allow the program to run on multiple cores with the default installation. So there is the multiprocessing library to get the program to run on multiple cores simultaneously. After this the program showed up on multiple cores when run and then the shared memory was tweaked to get the separate processes communicating.

The Idea

The idea is to take the hash of a file and then try to recreate the file with a known length of random ones and zeros. This works very well for very small file sizes but the brute force method works terribly for large sized files. The limit seems to be the number of random tries needed to achieve the reverse hash. If one had access to a supercomputer with thousands of cores this might make sense. Then again the question remains as to why you would need to compress something so heavily if you had access to that kind of hardware.