Multithreading/concurrent runing selenium Windows

M

Mercy

Guest
I'm currently working on a project with webparsing and using selenium together with phantomJS.

The project I wrote is working fine but the problem is that it is very slow, so i thought i might introduce multithreading in it.

I know that selenium is not multithread safe and each thread should have its own webdriver(just to be clear). The point is: I have an array with objects that are used in the url(only a part of it) Currently I running for loop for the objects and navigate the driver to given xpath, get data and initialize the object. To do it faster I want to create 4 threads where each thread should: 1. Initialize new web driver 2. Get the item from list that is not used yet. 3. Call different functions on this object

I'm not sure how I should start to create multiple drivers and put it all together. How I'm suppose to create the threads that are responsible for the work? I have read about the run override function and that st this place there should be the work that thread should do, but how it should know the next element from list to use?

Continue reading...
 
Top