Course Description: A study of both hardware and software issues connected with solving a problem in a parallel processing environment which may include grid computing, cluster computing, or special hardware configurations such as a muti-core processor. Emphasis will be placed on identifying the basic properties of bandwidth, latency, scalability and granularity as it relates to an algorithmic solution to a problem. Students will design, code, test and debug programs for stated environments.

Prerequisites: CSC-1720 Advanced Programming with Data Structures

Course Materials:

1. David B. Kirk, Wen-mei W. Hwu, Programming Massively Parallel Processors - A Hands-on Approach. 4th edition Morgan Kaufmann

2. Optional: Jason Samders, Edward Kandrot, CUDA by Example - An introduction to General-Purpose GPU Programming Addison Wesley

3. Time and patience.

4. Account on the Linus (A PC running Linux). Linus can be accessed through Internet by using the address linus.highpoint.edu. This will give you access to several machines that contains high perfromance GPUs from Nvidia. This is where I will develop and test all programs during class. The C-like code (nvcc) in this course will be generic and programs written in this class should work on any machine containing a GPU from Nvidia with Cuda/openCL installed.

Course Objectives: After successfully completing this course, you will be able to:

1. Compare and contrast architectures for parallel computing, recognizing the strengths and weaknesses of each.

2. Compare and contrast parallel programming paradigms recognizing the strengths and weaknesses of each.

3. Identify the basic properties of bandwidth, latency, scalability and granularity.

4. Design, code, test and debug programs for parallel computation.

Programming Assignments: In addition to the major tests and exam, students will participate in a series of programming assignments to further develop their logic problem solving skills. Each assignment will require the student to analyze a problem and decompose it into simpler tasks that is suitable for a parallel environment. At that point, the student will then implement, test, and debug a program that solves the problem.

1. STYLE 30%

   1. Three space indenting for each block of code. For the most part only one C/C++ statement per line. Lines end with a hard return. Functions are 30 lines or less - very few exceptions! Is the program readable on paper? Does indenting help debugging and readability? Are parts of the program clearly labeled?

   2. Identifiers. Are the identifiers meaningful and mnemonic? Do they describe their purpose?

   3. Documentation/Comments. Is the program reasonably commented? Does the commenting help both the programmer and the grader? Could another programmer read and understand the program?

   4. Attractive output?

2. PROGRAM 70%

   1. Algorithm. Is the program sensible? Does it solve the problem in a reasonable manner?

   2. Modularity. Does the program take advantage of subroutines and functions? Are its separate parts easy to isolate for testing and debugging? Again, if functions are longer than 30+/- lines, then rethink the algorithm.

   3. Working. Does the program work? Does it fulfill the problem description, or does it solve some other problem? Is it written in a way that it lets shortcomings be recognized and fixed?

   4. Use of required techniques and algorithms?

3. Late Penalty: Programming assignments will have due dates. You will automatically get an extension of 7 days if you miss the due date. The penalty will be 1 point per day late. No program will be accepted after 7 days of the assigned due date.

Course Policy:

Assessment: The course objectives will be assessed through major tests, the final exam and problem set assignments. The final grade will be determined as follows:

Programming Assignments: 40%

Beowulf Cluster Build: 15%

2-3 Major Tests: 30%

Cumulative Final Exam: 15%

Total: 100%

Range	93-100	90-93	87-90	83-87	80-83	77-80	73-77	70-73	67-70	63-67	60-63	0-60
Grade	A	A-	B+	B	B-	C+	C	C-	D+	D	D-	F

You must score at least the minimum of a category to be given the associated grade, i.e. an average of 80 is a B- not a C+.

8+ Programming assignments - You will receive the following grade for programming assignments based on the number of correctly solved problems. Adjustments from the scale may be made due to partial solutions, poor documentation, etc.

Solved	0	1	2	3	4	5	6	7	8
Grade	0	20	40	60	70	80	90	95	100

Major tests and Final Exam: All tests and final exam will consist primarily of problem solving type questions similar to the homework assignments and quizzes. No make up test will be given unless prior arrangements have been made with excusable reasons. If you miss a test your final grade will be used in place of the missed grade. This means the percentage of the final exam will increase. If you miss the final exam, a 0 will be placed in all missing tests.

Attendance: High Point University recognizes and honors the value of the face-to-face classroom experience for enhancing student engagement and academic outcomes. Attendance will be taken daily at the beginning of class. As such, you are expected to be present in every class meeting with the exception of an illness that requires quarantine or other reasons pre-approved by the instructor. In those cases online accommodations will be provided. If absent from class, you are responsible for all lectures, in-class exercises, assignments, and handouts. Caution: Being absent from class does not change the due date of assignments. Before a student is dropped for non-attendance, I will give a notification as a warning to a student that he/she is in danger of violating the attendance requirements.

Honor Code: All students are expected to follow the University Honor Code, Full details of the High Point University Honor Code are found here: http://www.highpoint.edu/studentconduct/university-honor-code All academic work should be completed with the highest level of honesty and integrity.

Professionalism: The Mission Statement of the Webb School of Engineering is: To provide an extraordinary engineering education through the delivery of a hands-on, practical education that prepares graduates for a rewarding career or admission into graduate school. The School of Engineering is committed to:

• Providing an environment that encourages curiosity.

• Developing technical competence within a Liberal Arts framework.

• Fostering independent thought and critical thinking to solve modern engineering and computing problems in an ethical, socially-responsible, and sustainable manner.

Please treat this class as if it were a company or organization for which you are working. You can exhibit professionalism by attending consistently, arriving on time, communicating your full or partial absence in a timely fashion, completing assignments on time, maintaining engagement, and respecting your peers and the professor.

Classroom etiquette:

• Cell phone usage during class is prohibited. Turn off cell phones (or silence mode) and place into backpack, i.e. remove phone from access during class.

• Do not listen to your digital media during class or lab time.

• Profanity will not be tolerated. Lewd remarks printed on your clothing are also not tolerated.

• Hats are not to be worn during class.

• Laptops are allowed during class and can only be used for class related instruction. Inappropriate use of a laptop, desktop or other electronic computing device during class will result in you being asked to leave the room.

Lab etiquette:

• The labs were created with student input. Please help respect and maintain the lab.

• Do not eat while sitting at a workstation. If you must eat while in the lab, sit a the larger table where no equipment may be damaged except your own. Clean up the area after you finish eating.

• Clean up your area before you leave the lab. Unless you are planning to return in a few minutes, please take all books, paper, etc with you.

• DO NOT lock your workstation when you leave!

Students with Disabilities: High Point University is committed to ensuring all students have equal access to all services and benefits at High Point University. If you are a student with a disability and require academic accommodations due to a diagnosed disability, you must register with the Office of Accessibility Resources and Services (OARS) and submit the appropriate documentation. Requests for accommodations should be made at the beginning of a course. Accommodations are not retroactive. Contact us at oars@highpoint.edu or by telephone at 336-841-9026, for additional information. The Office of Accessibility Resource and Services is located on the 4^th Floor of Smith. Library.

Face covering: Wearing a face mask reduces the risk of COVID-19 transmission and is a step we can all take to care for the members of our HPU family. This class will abide by the guidelines that are set forth by the University. Please note this may change as the semester progresses.

Course Topics:

1. Introduction - models of computation

   • SIMD

   • MIMD

2. Beowulf Cluster

3. Programming Models

   • openMPI

   • openMP/openACC

   • Cuda/OpenCL

4. Address-Space

   • UMA

   • NUMA

   • Message Passing Interface

5. Performance Analysis and Measurement-Problems with parallel environments

   • Bandwidth

   • Latency

   • Scalability

   • Granularity

6. Parallel architectures

   • Processor architectures including multi-core CPUs/GPUs

   • Memory systems for high performance

• Caching and coherence

• Clusters

• Grid computing

1. Parallel programming paradigms

   • Threading

   • Message passing

   • Event driven techniques