Milestone 2

Due: Thursday, November 19th at 11:59 PM

Submission Instructions

You will be writing code for this milestone. Please be sure that we can find it in your project organization.

Your submission should be in the following format:

project_repo/
             README.md
             docs/  
                  milestone1
                  milestone2
                  milestone2_progress
             code/
                 ...

Note that code should be named something informative. Note too that the documentation for Milestone 2 will be a superset of that for Milestone 1. Hence, it should be in a new file.

Requirements

Here are the main requirements for the second milestone:

1. Working forward mode implementation
2. Note: For this milestone, you are not required to support multiple functions of multiple inputs yet. All tests run by the teaching staff will only be run on scalar functions of a single input.
3. Test suite
4. Updated / extended documentation
5. Proposal for additional feature(s)

See the sections below for more specific details.

Working Forward Mode Implementation

You must have a working forward mode implementation. This does not mean, however, that you should have a complete implementation. Guidelines are in the following sections.

Minimum Package Requirements

• The software should be available for download from your GitHub organization.
• At this point, there is no need for you to release it on PyPI.
• You should provide a requirements.txt file (or something similar) with your software so other developers are able to install the necessary dependencies.
  • C++: Provide a Makefile and/or a build script.
• After a user installs your package, they should be able to use it without difficulty.

Minimum Implementation Requirements

The following is a description of a typical use case.

• A user downloads your package from your organization (pip not required yet).
• They install the dependencies and run the tests.
• They create a "driver" script in the top level.
• Note: How they interact with your package will depend on your implementation. The interface and other implementation details should be described in your documentation.
• The next few steps may sound somewhat abstract, but that is only because they hinge on your specific implementation.
• In the driver script, they import your package.
• They instantiate an automatic differentiation object to be used in the forward mode.
• They write a root-finding algorithm (e.g. Newton's method) that requires calculation of the Jacobian.
• They access the Jacobian via the automatic differentiation object.
• Note: For this milestone, the Jacobian will just be a scalar. You are not required to support multiple functions of multiple inputs yet. All tests run by the teaching staff will only be run on scalar functions of a single input.

There are other use cases as well. Someone may want to solve an optimization problem using the forward mode. Regardless, the workflow outlined above should be roughly the same.

What Kinds of Functions should be Implemented?

This document began by saying that you don't need to have the forward mode fully implemented. This section describes what that means.

Recall that eventually a user should be able to use your software to get derivatives of a vector function of a vector. You should keep this in mind when designing your software. For this milestone, however, the requirements are a little bit relaxed. A typical user this time will just want to calculate derivatives of a scalar function of a scalar. At a minimum, your software should overload all the basic operations (addition, multiplication, subtraction, division, power). Don't forget about the unary operations such as negation. It should also contain the following elemental functions:
- exponential
- trig functions (sine, cosine, tangent)

Feel free to go beyond the bare minimum. You'll have to do it eventually before the semester ends.

Test Suite

You should have a test suite that runs with pytest (or a similar C++ test framework like Google Test). Your test suite should run automatically on Travis CI. The project GitHub repo should contain a badge showing that your code is passing all of your tests.

You should also have your project connected to Codecov. Once again, the project repo should have a badge reporting on the coverage of your code from Codecov. The repository should reflect at least 90% code coverage. See An introduction to code coverage for a discussion on code coverage.

Documentation

Be sure to extend your documentation from Milestone 1. Now is a good time to think about the best way to distribute your documentation. You may want to consider using a Jupyter notebook with Markdown cells interspersed with code blocks for actual hands-on demos. The same thing can be achieved with a standard Markdown document, but it won't be as interactive.

You will receive full points as long as you have a docs/ folder and your documentation is complete. However, you may want to consider alternative ways of hosting your documentation. For example: Read the Docs or Sphinx. If you're using C++, you should consider Doxygen.

Documentation Sections

The teaching staff will use the documentation to run your package. They will implement a root-finding algorithm that requires first derivative information. See the section What Kinds of Functions should be Implemented? to get an idea of the types of elementary functions that could be used.

The following sections should be present:

• Introduction
  • Describe problem the software solves and why it's important to solve that problem. This can be the same as Milestone 1 with updates based on comments from Milestone 1 grading.
• Background
  • This can be the same as in Milestone 1 unless you were asked to update it.
• How to use your package
  • How to install? Even (especially) if the package isn't on PyPI, you should walk them through the creation of a virtual environment or some other kind of manual installation.
  • Include a basic demo for the user. Come up with a simple function to differentiate and walk the user through the steps needed to accomplish that task.
• Software organization
  • High-level overview of how the software is organized.
    • Directory structure
    • Basic modules and what they do
    • Where do tests live? How are they run? How are they integrated?
    • How can someone install your package? At this point, it is okay if your package isn't on PyPI. If it's not, then you should describe how someone can download and install your package manually.
• Implementation details
  • Description of current implementation. This section goes deeper than the high level software organization section.
    • Try to think about the following:
      • Core data structures
      • Core classes
      • Important attributes
      • External dependencies
      • Elementary functions
  • This is similar to what you did for Milestone 1, but now you've actually implemented it.
  • What aspects have you not implemented yet? What else do you plan on implementing?

Future Features

Now that you've got most of the hard implementation work done, what kinds of things do you want to impelement next? How will your software change? What will be the primary challenges to implementing these new features? Things you may want to consider here include (but are not limited to) any changes to the directory structure, and new modules, classes, data structures, etc.

We are checking for reasonable ideas here. If you really can't come up with anything interesting, you should consult the lecture notes. If you are still lost after that, please ask your TF or the instructor for help. If you are unsure about the viability or difficulty of your proposed extension, please ask your TF or the instructor for their thoughts. The extension should not be trivial and you should have given it sufficient thought that you can write something concrete about how to proceed. Please be creative and have fun with it!

A Note on C++

If you are doing a C++ project, you have two options for your final project. The first option is to write an AD library from scratch. You should follow the instructions outlined above to accomplish this. The second option is to add forward mode to an existing AD library. In this case, you should show the PR and Issue in the base code's repo and have a clear plan on how to address this issue. Make sure you contribute to the test suite and documentation. You should also be in touch with the developer community of that library. We want to see a clear plan of action. Even though you're not writing a library from scratch, you will still confront obstacles and technical hurdles that require a plan to navigate.

Grading Breakdown