Conv3D

[BRevolution]

Hi,
I'd like to use your library with models that are trained on 3d images. How hard do you think would it be for me to add support for 3d convolutions?

best regards

[Dobiasd]

From the point-of-view of the library infrastructure it wouldn't be a big problem. It's just new layer type.
The challenges would be:

• actually implement this layer and the calculations in a performant way
• exporting the weights correctly in Python, i.e. extending adjusting convert_model.py

But with adding appropriate tests in generate_test_models.py one could be quite certain to have implemented it correctly, once the tests pass.

[Dobiasd]

Are you interested in giving it a shot?

[BRevolution]

Well, maybe. Let's see if I can get it to compile on my machine first^^

[Dobiasd]

@BRevolution How is it going? Any news on that? :)

[BRevolution]

Hi Dobiasd,
well, I searched for other solutions and decided to use tfcompile instead. I got it to work with it, but it turned out to be super slow and not even multithreaded. So I'm basically back to square one :-/.
I still hestitate to implement 3d convolutions in your project because I would also need to implement multithreading to make it run fast.

[Dobiasd]

would also need to implement multithreading to make it run fast
Depends on if you need:

• throughput (predictions per second) or
• "latency", in a sense of minimal runtime for a single prediction

The throughput issue could be solved by just running multiple independent predictions in parallel (fdeep::model::predict_multi).

[Dobiasd]

tfcompile [...] turned out to be super slow

That somewhat surprises me. All compiler optimization etc. were switched on?

[BRevolution]

Sadly I just need to do a single prediction with batchsize 1. The CPU Tensorflow Python version needs about a minute (and uses all cores). The code generated by tfcompile needs about one hour (!), and yes, I turned optimizations on, including AVX2.

[Dobiasd]

Wow, that's extremely slow. Have you tried opening an issue in the TensorFlow repo?

[BRevolution]

I doubt that would solve anything. It's been quite a while since they added features to AOT and they seem to have no interest in adding multithreading. So even if they fixed the slowdown with singlethreading, it still wouldnt be fast enough.

[Dobiasd]

@BRevolution Are you still interested in Conv3D for frugally-deep?

[BRevolution]

If you were to implement it, I would gladly try it out. But I fear I do not have the time to implement it myself.

[Dobiasd]

OK. I'll keep it as a todo (and leave this issue open), but currently, I can not predict when I start working on it.

[ashishmagar600]

Any updates on the support of Conv3D?