To help you get started with the IFT6266 class projects, we’ll provide a snippets of code to read and process the datasets that you are asked to work with.
Dogs vs. Cats
The Dogs vs. Cats dataset was originally part of a Kaggle competition. The competition page and forums will have more details on the dataset and approaches that people have tried. Also have a look at the leaderboard from last year’s class.
Fuel is a data-processing framework for machine learning that helps you download, read and process data for the training of machine learning algorithms. Fuel contains a wrapper for the Dogs vs. Cats dataset. To get started with it, begin by following the installation instructions. Afterwards, follow the instructions for downloading a built-in dataset but use dogs_vs_cats
instead of mnist
. This will download the Dogs vs. Cats dataset for you and convert the JPEG images into a numerical HDF5 file (which can easily be viewed as NumPy arrays). The following commands should get you started:
cd $HOME
mkdir fuel_data # Create a directory in which Fuel can store its data
echo "data_path: \"$HOME/fuel_data\"" > ~/.fuelrc # Create the Fuel configuration file
cd fuel_data # Go to the data directory
fuel-download dogs_vs_cats # Download the original dataset into the current directory
fuel-convert dogs_vs_cats # Convert the raw images into an HDF5 (numerical) dataset
Fuel was written to deal with out-of-memory datasets, streaming data, parallel on-the-fly preprocessing of data, etc. in mind, so take some time to read the overview in order to understand the basic terminology. As a quick pointer, consider the following example:
# Let's load and process the dataset
from fuel.datasets.dogs_vs_cats import DogsVsCats
from fuel.streams import DataStream
from fuel.schemes import ShuffledScheme
from fuel.transformers.image import RandomFixedSizeCrop
from fuel.transformers import Flatten
# Load the training set
train = DogsVsCats(('train',), subset=slice(0, 20000))
# We now create a "stream" over the dataset which will return shuffled batches
# of size 128. Using the DataStream.default_stream constructor will turn our
# 8-bit images into floating-point decimals in [0, 1].
stream = DataStream.default_stream(
train,
iteration_scheme=ShuffledScheme(train.num_examples, 128)
)
# Our images are of different sizes, so we'll use a Fuel transformer
# to take random crops of size (32 x 32) from each image
cropped_stream = RandomFixedSizeCrop(
stream, (32, 32), which_sources=('image_features',))
# We'll use a simple MLP, so we need to flatten the images
# from (channel, width, height) to simply (features,)
flattened_stream = Flatten(
cropped_stream, which_sources=('image_features',))
Note that the Dogs vs. Cats dataset only has a training set and a test set; you’ll need to create your own validation set! This is why we selected a subset of 20,000 images of the 25,000 as our training set.
You’ll need to extend this example a bit for it to work well; 32 x 32 crops are too small, but some of the images aren’t any bigger. To deal with this you’ll need to upscale the smaller images (see e.g. fuel.transformers.image.MinimumImageDimensions
), but you might also want to downscale the bigger ones.
You can download the original datasets from Kaggle or from here: train, test.
Quick example with Blocks
Blocks is a framework for training neural networks that is used a lot at MILA. It helps you build neural network models, apply optimization algorithms, monitor validation sets, plot your results, serialize your models, etc.
Blocks is a framework made for research, and isn’t as plug-and-play as some other frameworks (e.g. Keras). As such, we don’t expect you to use it. However, some parts such as the optimization algorithms and monitoring can save you some time. Have a look at Blocks’ introduction tutorial to get you started and consider the following example which shows you how to optimize a Theano expression of a cost using Blocks.
# Create the Theano MLP
import theano
from theano import tensor
import numpy
X = tensor.matrix('image_features')
T = tensor.lmatrix('targets')
W = theano.shared(
numpy.random.uniform(low=-0.01, high=0.01, size=(3072, 500)), 'W')
b = theano.shared(numpy.zeros(500))
V = theano.shared(
numpy.random.uniform(low=-0.01, high=0.01, size=(500, 2)), 'V')
c = theano.shared(numpy.zeros(2))
params = [W, b, V, c]
H = tensor.nnet.sigmoid(tensor.dot(X, W) + b)
Y = tensor.nnet.softmax(tensor.dot(H, V) + c)
loss = tensor.nnet.categorical_crossentropy(Y, T.flatten()).mean()
# Use Blocks to train this network
from blocks.algorithms import GradientDescent, Scale
from blocks.extensions import Printing
from blocks.extensions.monitoring import TrainingDataMonitoring
from blocks.main_loop import MainLoop
algorithm = GradientDescent(cost=loss, parameters=params,
step_rule=Scale(learning_rate=0.1))
# We want to monitor the cost as we train
loss.name = 'loss'
extensions = [TrainingDataMonitoring([loss], every_n_batches=1),
Printing(every_n_batches=1)]
main_loop = MainLoop(data_stream=flattened_stream, algorithm=algorithm,
extensions=extensions)
main_loop.run()
First things to try
- You’ll want to use convolutional nets.
- There is currently no transformer in Fuel that resizes images to a fixed size (i.e. make sure that the shortest size is N pixels). This might be a good idea to implement.
- Resizing and cropping images can be CPU-intensive. Moreover, it is a good idea to to data augmentation on this dataset e.g. add rotations, distortions, etc. to make sure the network doesn’t overfit. By default these operations are done in the same process that controls the training on the GPU (or CPU) which means that during the data processing no training is happening and vice versa. This isn’t very efficient! Fuel has a “data server” which allows you to run training and data preprocessing/augmentation in parallel. Have a look at the tutorial here.
Vocal Synthesis Project: Spiritual Ascension Music
The second dataset is a 3 hour long audio track from a YouTube video. For this too a Fuel wrapper is available, but you’ll need to make sure to install the Python module pafy
(use pip install pafy
) and the ffmpeg
package.
On Linux you can probably use e.g. sudo apt-get install ffmpeg
or yum install ffmpeg
depending on your platform, while on OS X it’s probably easiest to use Homebrew (brew install ffmpeg
). There are also Windows builds available.
fuel-download youtube_audio --youtube-id XqaJ2Ol5cC4
fuel-convert youtube_audio --youtube-id XqaJ2Ol5cC4
If you can’t manage to install ffmpeg
, you can also download the HDF5 file directly from Dropbox and simply place it in your Fuel data path. (The WAVE file is also available.)
from fuel.datasets.youtube_audio import YouTubeAudio
data = YouTubeAudio('XqaJ2Ol5cC4')
stream = data.get_example_stream()
it = stream.get_epoch_iterator()
sequence = next(it)
Note that this gives you the entire sequence as one batch. During training, you probably want to split up the sequence in smaller subsequences e.g. of length N, to avoid running out of memory.
To do this you will need to implement a Fuel transformer.
Tips
- Even if you don’t use Blocks in order to construct your model, be sure to look at its implementations of e.g. the LSTM and GRU units; it’s easy to get wrong!
- If you want to get started on a Fuel transformer that produces sets of subsequences, have a look at the
NGrams
transformer. Discussion on how to do this exactly is ongoing on this pull request.
If you’re using Blocks, be sure to manually upgrade Theano (to version 0.8) via git. If you’re using Theano 0.7 from Anaconda, batch_normalization is not implemented yet, causing some errors from bricks.
LikeLiked by 1 person
If you followed the installation instructions precisely it should have updated Theano to the bleeding edge version from GitHub automatically. But yes, Blocks is developed and tested with the latest Theano only.
LikeLike
It seems that the cats and dogs dataset is not one of fuel’s built-in datasets ( it is not there when I typed in the command fuel-download -h). What should we do in this case ?
LikeLike
I added it to Fuel very recently, so please make sure you installed the latest version from GitHub. Have a look at the installation instructions (and make sure you didn’t just do
pip install fuel
, that will install a version that is too old).LikeLike
Never mind. You just have to download the cutting edge version of fuel to get the dataset.
LikeLike
One can read that ”there is currently no transformer in Fuel that resizes images to a fixed size (i.e. make sure that the shortest size is N pixels)”, but there is transformer called RandomFixedSizeCrop which can do some random crops on images. I’m not sure to see what is meant by ”resize” if it is not about cropping. Do you mean resizing all images to smallest image size? Thanks!
LikeLike
RandomFixedSizeCrop
will take a random crop of the image. This crop needs to be smaller than all images e.g. you cannot take 64 x 64 crops if there’s a single image that is 63 x 80.Luckily there is the
MinimumImageDimensions
transformer, which will upscale smaller images to have a minimum size. However, it won’t downscale larger pictures. That means that for small images your crops are very large and will probably contain a large part of the dog or cat (which is probably what you want). However, since the large images weren’t downscaled, your 64 x 64 crop might only contain a whisker, ear or background for a 2000 x 2000 picture.Hence, it’s worth considering scaling all your images so that the smallest side is N pixels, and then take crops of M x M (where M <= N). This way your crop is most likely to contain a large part of the cat/dog.
LikeLiked by 1 person
I do not know after I download youtube files (it is ok), but when I am trying to convert it, I get error: fuel-convert youtube_audio –youtube-id XqaJ2Ol5cC4
Traceback (most recent call last):
File “/home2/ift6ed05/anaconda2/bin/fuel-convert”, line 9, in
load_entry_point(‘fuel==0.1.1’, ‘console_scripts’, ‘fuel-convert’)()
File “/home2/ift6ed05/anaconda2/lib/python2.7/site-packages/fuel/bin/fuel_convert.py”, line 69, in main
output_paths = convert_function(**args_dict)
File “/home2/ift6ed05/anaconda2/lib/python2.7/site-packages/fuel/converters/youtube_audio.py”, line 38, in convert_youtube_audio
ffmpeg_not_available = subprocess.call([‘ffmpeg’, ‘-version’])
File “/home2/ift6ed05/anaconda2/lib/python2.7/subprocess.py”, line 522, in call
return Popen(*popenargs, **kwargs).wait()
File “/home2/ift6ed05/anaconda2/lib/python2.7/subprocess.py”, line 710, in __init__
errread, errwrite)
File “/home2/ift6ed05/anaconda2/lib/python2.7/subprocess.py”, line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
LikeLike
It seems the error refers to missing ffmpeg. Did you get ffmpeg installed?
LikeLike
Thanks, I forgot to install ffmpeg, as I am using a machine which I do not have right to sudo or as a root, it gives me error when I am using yum install ffmpeg:Loaded plugins: refresh-packagekit
You need to be root to perform this command.
And I can not do sudo apt-get insatll ffmpeg.
LikeLike
you could download and install ffmpeg into your home folder, this way you have permissions over the files.
Then add the line ‘export PATH=”$PATH:$HOME/ffmpeg/bin” ‘ without ‘ quote to the end of ~/.bashrc, then re-login terminal
Let me know if it works
LikeLike
After I do make, I get this error
make
Makefile:2: config.mak: No such file or directory
Makefile:62: /common.mak: No such file or directory
Makefile:104: /libavutil/Makefile: No such file or directory
Makefile:104: /library.mak: No such file or directory
Makefile:106: /doc/Makefile: No such file or directory
Makefile:189: /tests/Makefile: No such file or directory
make: *** No rule to make target `/tests/Makefile’. Stop.
LikeLike
These errors seems to suggest that it simply can’t find the
Makefile
. Did you run the./configure
command successfully? Are you sure you successfully didcd ffmpeg-2.8.6
?Note that if you can’t manage to install Ffmpeg you can always just download the HDF5 file directly.
LikeLike
After I run ./configure, I get this error, yes I am in that folder, and where can I download hdf5 file directly?
./configure –prefix=”$HOME/ffmpeg” –disable-yasm
Unknown option “–prefix=”/Users/chendanlan/ffmpeg””.
See ./configure –help for available options.
LikeLike
Note that the flag is
--prefix
and not-prefix
(WordPress probably turned--
into –).The link to the file is both in the “Getting Started” instructions, and in the comment I just made.
LikeLike
Oh, I am sorry, and it is also –disable. Thanks very much @Bart and @Melvin
LikeLike
Could we use other package like lasagne? Or we must use blocks?
LikeLike
As said on the Getting Started page:
You can use any package you want, or not use a package at all (and just write Theano).
LikeLike
@Bart, Hello Bart, I have a question about installing anaconda or any other software in the remote machine at calculquebec.ca. I don’t know if the anaconda or numpy+theano is already installed in the student accounts. Because when I type python, in the terminal and import numpy or anaconda, the python responded that these packages are not installed. When I try sudo apt-get install xxxx. It said that I should use sudo very responsibly, but at the end it returned error saying that I’m not o n the sudo-er list…Finally I downloaded the source code and uploaded it on the remote machine and tried python setup.py install, it tried to compile but at last told me error: could not create ‘/usr/lib64/python2.6/site-packages/numpy’: Permission denied.
I would like to know how do we install software (especially anaconda and then…) on those remote machines? Do we have sudo rights? Thanks
LikeLike
Install anaconda using bart’s video instruction. Follow it exactly. It should be installed in your /home/ dir. you can check which python you’re using with the command
which python
it should give you a path to your anaconda install in your own dir. eg.
/home/USERNAME/anaconda/bin/python
LikeLike
how do we “save” our output from our classification? in a hdf5 file? if so, do we need to convert the hdf5 file back to an audio file?
LikeLike
We use HDF5 files to store datasets because they have efficient tools for e.g. caching when things don’t fit into memory. For your samples/predictions you can use whatever format is easiest for you e.g.
.npz
or.npz
files work as well.If you think your samples are good, then of course it would be interesting to convert it back into a wave file to see what it sounds like. As a first step plot your samples though and see if they look like wave signals at all.
LikeLike
Hi Melvin,
I used scipy.io.wavfile
from fuel.datasets.youtube_audio import YouTubeAudio
import scipy.io.wavfile
data = YouTubeAudio(‘XqaJ2Ol5cC4’)
stream = data.get_example_stream()
it = stream.get_epoch_iterator()
sequence = next(it)
newsample = sequence[0][:60000]
scipy.io.wavfile.write(“newsample.wav”, 44100, newsample)
LikeLike
You might also want to make sure that your newsample’s data type is int16 or else you will have trouble recognizing the sound because or distortion.
LikeLike
It would actually beb better to use a sampling rate of 16MHZ, so you should use scipy.io.wavfile.write(“newsample.wav”, 16000, newsample)
LikeLike
Is there a reason to use 16khz (I think you mean kHz not MHz?) Instead of 44100?
LikeLike
The audio data was originally downsampled to 16 kHz (see the Fuel code) to keep the timescale more reasonable. If you’d save it as 44.1 kHz I imagine things would sounds like they’ve been sped up 3 times.
LikeLike
It sounds exactly like that.
LikeLike
How can we interact with an dataset object to get to get the names of the sources. I tried DogsVsCats.sources but it returns an inscrutable answer
Let’s say I have imported the dataset
from fuel.datasets.dogs_vs_cats import DogsVsCats
and I want to find find out what are the sources available and how they are called. That is I want to have a summary of the data (what are the dimensions and names of the datasets and so on ). I couldn’t find any documentation on this.
LikeLike
You need to check the
.sources
attribute on the instance, not on the class:If you want access to the actual HDF5 handle you’ll need to dig a little deeper. You’ll find that the images are stored as a list of vectors with their shapes stored separately.
LikeLike
Once we have defined a data stream, how can we interact with the data generated from the stream? I have written a stream that upscales images as they comes, based on the introductory code, but am having difficulty generating images from the stream.
This is my code:
Load the training set
train = DogsVsCats((‘train’,), subset=slice(0, 20000))
We now create a “stream” over the dataset which will return shuffled batches
of size 128. Using the DataStream.default_stream constructor will turn our
8-bit images into floating-point decimals in [0, 1].
stream = DataStream.default_stream(
train,
iteration_scheme=ShuffledScheme(train.num_examples, 128)
)
Our images are of different sizes, so we’ll use a Fuel transformer
to upscale the smaller images to be of minimum size (512 x 512)
upscale_stream = MinimumImageDimensions(stream, (512,512), which_sources=(‘image_features’,))
Let’s say I want to load the first batch of 128 examples to check whether upscaling is performed correctly,how would I go about doing that?
Also, how can we format python code snippets nicely in WordPress blog entries ?
LikeLike
I enabled Markdown support for formatting Python code.
Please have a look at the Fuel documentation. You could use something like
iterator = upscale_stream.get_epoch_iterator()
and thenbatch = next(it)
to get the first batch.Note however that what you’re trying to do won’t work.
Datastream.default_stream
will turn the images into floating point numbers, andMinimumImageDimensions
doesn’t like that. For a detailed explanation, see this issue.LikeLike
Anyone currently working on vocal synthesis? (Or plan to?)
LikeLike
Just started. Nothing to report so far.
LikeLike
I am working on vocal systhesis too.
LikeLike
Generated my first audio, woot!! https://soundcloud.com/christopher-beckham-134376575/3-layer-lstm-with-600-hidden-units
Will post details on my blog soon: https://cjb60.wordpress.com/
LikeLike
It seems that your sound is distorted. To many channels perhaps?
Did you try to convert your raw data type into int16 before converting it wav?
newsample = yhat.astype(np.int16) #where yhat is the sample to convert
scipy.io.wavfile.write(“newsample.wav”, 16000, newsample)
LikeLiked by 1 person
Oh! it seems you already know how.
Nice work.
LikeLike
Thanks anyway! Check my 2nd blog post, with that fix I was able to generate legible sounding audio 🙂
LikeLike
Just made a blog post that goes into some detail about what I did: https://cjb60.wordpress.com/2016/03/08/a-write-up-of-what-i-have-done/
LikeLiked by 1 person
Did you have any problems with the generated audio being too soft? I had to artifically boost the amplification using audacity, then I could hear something.
LikeLike
No I haven’t had this problem. It looks like you’re saving the output .wav file correctly, which is good, but I noticed that your plot of the generated music has amplitudes between -1500 to +1500, whereas in the original sound it’s between about -6000 and +6000. What kind of feature scaling have you done? Maybe it’s related to that. 🙂
LikeLike
Just made a totally rookie mistake (super embarassing because my “generated” audio was also played in class!) and found that the music that was played was actually just the seed sequence and not the seed sequence + the audio that my LSTM was generating! It seems like my LSTM still cannot generate something that sounds even remotely like the actual audio. I must press on and keep working on this 🙂
LikeLike
I encountered this problem : when I run the following [python code] (https://github.com/yiulau/ift6266/blob/develop/server.py “Title”)
I get this [error message] (https://github.com/yiulau/ift6266/blob/develop/nohup.out “Title” ) .
LikeLike
This looks like you didn’t install Fuel correctly. Upon installation there are Cython extensions that should have been compiled, which didn’t seem to have happened here. Try reinstalling Fuel, or if you installed it by just adding to
PYTHONPATH
then build the Cython extensions in-place.LikeLike
Did anyone working on the DogsVsCats project encounter something similar ? Can you try to explain what is happening?
I set up a data stream to get batches of images from DogsVsCats. Then I used the following code to time how much time it takes to get a batch from the stream over an epoch.
And I got something like this as time between each batch:
0.115445137024
0.0785758495331
0.066997051239
0.0754771232605
0.0781049728394
0.0756139755249
0.0682671070099
0.087070941925
0.0688378810883
0.0686841011047
0.0885288715363
0.0747940540314
0.0728580951691
0.0670669078827
0.0878319740295
0.0848870277405
0.0787830352783
0.0777719020844
0.0734760761261
7.66090202332
12.0970921516
13.2528531551
13.7659142017
13.7945039272
11.2318739891
12.5979430676
At the beginning of the iteration it was fast and then all of a sudden it hit a wall and became really slow.
I am not using DataServerStream so there shouldn’t be any buffer issue there.
LikeLike
This question is nearly impossible to answer without more information.
Are there other processes running that could be in the way? Are the images of the slow batches bigger perhaps, so that resizing and/or cropping takes more time? Are you running out of memory, causing your computer to start swapping?
You could try running your script with
python -m cProfile -o profile.log <script>
and then usepstats
to check the results to see what function exactly is being slow. More information is in the Python documentation.LikeLike
I got rid of the slowdown by moving fuel_data to /scratch.
Now I encountered another problem. When I am getting data from machine A it takes around 0.2 seconds per batch, but when I run the same script to get data from machine B it takes almost 10 times as much time; and I am not running a data server. How do I diagnose the reason for the difference ? It seems like both machines have about the same resources available.
On machine A the cpu is
model name : AMD Opteron(TM) Processor 6274
On machine B the cpu is
model name : Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
LikeLike
The Doc for class blocks.bricks.conv.ConvolutionalSequence(*args, **kwargs) says that parameter layers (list) – List of convolutional bricks (i.e. Convolutional, ConvolutionalActivation, or Pooling bricks) can be inputted (https://blocks.readthedocs.org/en/latest/api/bricks.html#blocks.bricks.conv.ConvolutionalSequence)
When I try to import ConvolutionActivation though I get ImportError: cannot import name ConvolutionalActivation. How can I use ConvolutionActivation?
LikeLike
I noticed that I can from blocks.bricks.conv import Activation without error. Do we have Doc on Activation. Can I use Activation as a layer in ConvolutionalSequence
LikeLike
https://groups.google.com/forum/#!topic/blocks-users/Y3BRXRtdhQ0
LikeLike
Just double checking… for the dogs vs cats project, the testing set is useless for the purpose of evaluating our model since it seems to not be labelled… we should then split the training set into training, validation and testing data?
LikeLike
You can obtain results on the test set by submitting your predictions on the Kaggle page.
Note that theoretically speaking you should only have to evaluate on the test set once at the end of your project. Any other evaluation that is used for model evaluation should be done on the validation set (which you should create yourself by taking a subset of the provided training data).
LikeLiked by 1 person
Has anyone gotten the following error when using Random2DRotation fuel image transformer (my code works with other fuel transformers):
LikeLike
Is it perhaps this issue? The
Random2DRotation
transformer requires images as inputs, so you can’t convert the data to floating point numbers. TheDogsVsCats
dataset does this when you use thedefault_stream
constructor, so if you want to use these transformers you can’t use it. You need to use theDataStream
constructor directly and scale the data manually afterwards using theScaleAndShift
transformer.LikeLike
Any hints on how to solve this problem?…
TypeError: Variables do not support boolean operations. This can happen if you do a logical operation between a numpy.ndarray and a Theano tensorvariable. Due to NumPy implementation before NumPy 1.8, we
cannot make the Python syntax work when the ndarray is on the left, and this results in this error. To work around that,
either call theano.tensor.{lt,le,eq,ne,gt,ge}(ndarray, tensor), or use the Python syntax with the Theano tensor on the
left. Or update to NumPy 1.8 or above.
I have python 1.10.4
Thank you!
LikeLike
BTW, the operation is this…
validation_frequency = min(n_train_batches, patience // 2)
LikeLike
Statistical questions:
I’ve split the audio data into train, test, and val sets, and subtracted the mean of the training set from each of them.
I normalized the training set based on the range of the training set, and for the test and val sets, normalized on whichever range was larger between it and the train.
In this case and in general, say for the test set, would it make sense to subtract something like the mean(mean_train, mean_test), and normalize based on
(And correspondingly for the validation)?
So you’re not leaking information from the test/val to the train, and you’re taking information about the train into account, but also considering that there may have been some bias in your test/validation data that you want to correct for. Likewise for the range; instead of taking the larger of the two as I did, is there some more intelligent way to do both of these things?
Maybe it doesn’t really make a difference; these preprocessing steps just feel hacky to me.
LikeLike
I’ve actually always just used the mean and variance of the training set to normalize the validation and test set. I actually have no idea what the convention is on using the statistics of the validation/test sets directly, but part of me feels it’s cheating a bit: In a real-world setting you might get the test samples one at a time, so you wouldn’t be able to calculate statistics over the entire test set before classifying your first sample.
LikeLike
Something interesting going on here:
when I uploaded my generated audio to youtube, youtube flagged it and gave me a copyright violation notice of the original Spiritual Ascension training audio. Does this mean that youtube’s algorithm matched my generated audio to be “similar” to the training audio? which mean I might be heading in the right direction? 😛
LikeLike
test
LikeLike
test2
LikeLike
I have a question, but my post is not showing up after sending it.
LikeLike
I’ve been trying to manage to use the lenet convolutional neural network code (deeplearning tutorial). For some reason, training is only possible when the feature dimension used is 784, same used for mnist. Attempting to use any other dimensionality will provide – ValueError Shape mismatch – x has 16200 cols (and 500 rows) but y has 800 rows (and 500 cols). The same error occurs independently of the the dataset (Mnist or DogsvsCats seem to raise the same issue). Any thoughts on how to remove this constraint regarding the feature dimensionality?
LikeLike
I am trying to view results but I can’t!!
While using
extensions.append(Checkpoint(“CatVsDog_model1.pkl”, after_epoch=True, after_training=True, save_separately=[‘log’]))
I am getting
Original exception:
TypeError: can’t pickle generator objects
Has anyone else gotten this error?
Thanks
LikeLike