Skip to content

Latest commit

 

History

History
73 lines (57 loc) · 3.19 KB

18647.md

File metadata and controls

73 lines (57 loc) · 3.19 KB
Raw

18-647: Computational Problem Solving for Engineers

Category Difficulty (out of 5)
Homework Assignments 3
Blog Post 3

This course guides students in solving AI/ML and engineering R&D challenges across diverse computing platforms, emphasizing productivity and performance. Covering a broad array of computational problems, from numerical simulation to AI, the course delves into critical topics such as scalable parallel algorithms, algorithm analysis, parallelization strategies, optimization, and result evaluation, providing students with a comprehensive skill set for tackling complex computational and informatics tasks.

Topics Covered

  • Computer architecture
  • Software stack
  • Numerical linear algebra
  • Algorithm analysis, scalabilty and complexity
  • Parallelization
  • Code optimization
  • Cloud computing and HPC
  • Numerical analysis
  • Data
  • Scalable algorithms
  • Modern CPUs
  • Hardware acceleration
  • Productivity to performance

Machines

In this course we use a large variety of machines in the homeworks and projects:

  • Large Linux servers with direct ssh access. The machines we access are the ECE number cluster and the data science cloud.
  • Large number of small Linux machines in high throughput mode via ECE's HTCondor.
  • HPC multicore nodes and multi-node configurations with and without GPUs via a batch system at PSC and CMU.
  • Hadoop/Spark/Tensorflow with and without GPUs at PSC.
  • Special hardware at PSC: Bridges AI multi-GPU resources (HPE Apollo and DGX-2) and large memory machines.
  • AWS instances with or without multicore and GPU support.

Software Platforms

In the course we try out a number of high-level languages and libraries in these languages:

  • Python with NumPy, SciPy, and matplotlib
  • R with CRAN packages
  • Matlab
  • Mathematica
  • Haskell
  • Frameworks: Torch, Theano, OpenCV
  • High-end engineering software
  • C/C++ with HPC math/ML libraries

Algorithms

We use algorithms covered in the lecture as simple examples to experiment across the machines and software platforms. Students either implement the algorithms themselves or find implementations online, and then run them on the target machines to perform scalability studies. Algorithm groups targeted in the homeworks are as following:

  • Dense numerical linear algebra, CNNs/DNNs/FFTs
  • Graph algorithms and sparse numerical linear algebra
  • ODE and PDE solvers, stencils, filters, discretization
  • Discrete and continuous optimization, ML training
  • Informatics, symbolic computing, higher level AI algorithms
  • Statistics: Monte Carlo, MCMC, statistical machine learning
  • Deep Learning, Big Data Analytics

Logistics

  • Two lectures per week.
  • No exams.
  • Grading will be based on homeworks, blogpost, and class participation.
  • This class is traditionally offered in both Fall and Spring.

Lectures

Taking into consideration the different backgrounds of the students in the lecture, the course is organized to gradual introduction for each topic.

How to do well

  • Pay attention to lectures and ask questions.
  • Attend the recitations every week, where the homeworks and the related concepts are well explained.