DeepLearning4J (DL4J) is a powerful Open Source distributed framework that brings Deep Learning to the JVM (it can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers). It can be used on distributed GPUs and CPUs. It is integrated with Hadoop and Apache Spark. ND4J is a Open Source, distributed and GPU-enabled library that brings the intuitive scientific computing tools of the Python community to the JVM. Training neural network models using DL4J, ND4J and Spark is a powerful combination, but it presents some unexpected issues that can compromise performance and nullify the benefits of well written code and good model design. In this talk I will walk through some of those problems and will present some best practices to prevent them, coming from lessons learned when putting things in production.
I am currently Regional (EMEA) Associate Director at MSD Biotech and was previously at Optum (UnitedHealth Group) and am based in Dublin, Ireland. I worked previously at IBM Ireland, where I switched my career path from Test Automation to Analytics and Machine Learning. I am passionate about coding, Big Data, AI/ML/DL, test automation, Open Source, DevOps and cooking (homemade pizza is my speciality!). I share my tech thoughts via my blog (http://googlielmo.blogspot.ie/) and DZone (https://dzone.com/users/2532948/virtualramblas.html). My first book "Hands-on Deep Learning with Apache Spark" (https://tinyurl.com/y7d98s64) has been released in January 2019.