Homework 1¶

Due Tuesday, September 15th 2020 at 11:59 PM.¶

Topics¶

Part 1: Course Survey¶

• Problem 1: Course Survey

Part 2: Basics¶

• Problem 2: Updating Github
• Homework Group: Check-In
• Problem 3: Unix
• Problem 4: Markdown
• Problem 5: Git

Part 1 [10 pts]: Course Survey¶

Problem 1 [10 pts]¶

Take the following survey: CS107 Introductory Survey.

Part 2 [90 pts]: Basics¶

Before proceeding, we recommend that you set up a nice directory structure for the semester. At a minimum, you should have a directory called Classes/ with subdirectories for each class you're taking this semester. One of these directories should be CS107 (or AC207 or CSCI-E-207). Inside CS107/ you will set up your course repo in Problem 2 below.

Problem 2: Updating GitHub¶

WARNING: If you don't do the following we won't be able to grade your assignment. There won't be points assigned to this problem but failure to do so can result in a 0 overall.

Note. Please make sure you have a Github Pro account. Request for one here: https://education.github.com/discount_requests/new.

Please do the following:

1. Make sure your course repo is named cs107_firstname_lastname. Then use the command git clone *url to your repo* in terminal to clone the GitHub repo to local. If you have already done this, then you're done and you can skip to Step 2. If you named the GitHub repo wrongly and cloned the repo with the wrong name to local, here's how you can rename your repo:
   
   1. Go to your GitHub repo and click on the Settings tab. Change the name under Repository name.
   2. Now go to the local copy of your repository. For the sake of the example, say that you called this cs107work.
   3. Type git remote set-url origin *url to your repo* where *url to your repo* will be something like https://github.com/github_username/cs107_firstname_lastname.git. What you just did was change the url of the remote repository. Recall that origin is the short name of your remote repository.
   4. As a sanity check, type git remote -v. It should list the correct url now.
   5. Finally, your directory should be updated to avoid any future confusion.
      • To navigate up one directory, type cd ...
      • Type mv cs107work cs107_firstname_lastname. You just renamed the directory!
      • Note that you should replace cs107work with whatever you originally named the repo.
        
        
2. Give the teaching staff access to your private repository:
   
   1. Go to your private repository again on GitHub.
   2. Go to Settings again.
   3. Click on Collaborators on the left side.
   4. Add the CS107 account username:
      • cs107-sys-dev
        
        
3. Now you're ready to create your first commit to the master branch. You will do everything from the command line. Set up a homework/ directory with the following steps (assuming you're in the root directory of the repository):
   1. mkdir homework/
   2. cd homework/
   3. echo "# Homework Directory" > README.md
   4. git add README.md
   5. git commit -m "Setting up homework directory."
   6. git push

Good job! You did everything from the command line instead of using your mouse! This is a very important skill to have.

WARNING: This is likely the only time you will make a commit directly to the master branch.

These steps will become second nature to you after this homework and the next few lectures. What you've done here is create a homework directory using some common Unix commands, generated a README file also using a common Unix command, and followed the usual git workflow of staging (via git add), commiting changes to your local repo (via git commit), and pushing to the remote repo (called origin). Note that the -m option to git commit allows you to specify a commit message directly from the command line rather than via your system's default text editor (e.g. vim).

Homework Branch Setup

Now that you've created your repo, please do the following to submit your HW1.

1. From your master Git branch, create a new development branch called HW1-dev.
   • Use the git checkout -b HW1-dev command to create a new local development branch.
   • We will only be grading the work you do on this development branch. You will lose 5 points if you do not do your development on the HW1-dev branch. As a token of good will, we will not take these 5 points off for this first homework. You are free to make other branches off of HW1-dev, say for individual problems, but you are responsible for making sure you merge these additional branches into the HW1-dev branch.
2. Within your homework/ directory, create subdirectory HW1/.
3. Under the HW1/ directory, create subdirectory HW1-final/.
4. Only files within HW1-final/ will be graded.
5. Follow the steps on the Coursework page to add files and create a pull request.

PR Merge Policy

• You will notice comments on your open PR while the TFs are grading. DO NOT merge the PR until the Canvas grades are released and available to you! Your homework grade is subject to change until that point. DO NOT ask TFs questions until the Canvas grades are released. Until that point, they are actively grading.

Note. Work from Problems 3 and 4 will be on this repo. Problem 5 is separate.

Problem 3: Unix¶

For these problems you will submit the single Unix command that outputs the desired result. Note that there are many different ways to do these problems. A short example of the absolutely essential Unix commands can be found at the end of this notebook: Some essential Unix commands.

• WARNING: We recommend that you do this problem in the provided Docker container mentioned in Part 3A. There can be variablility in outputs depending on the operating system you are on, but the container solution allows everyone to use a common OS. We will grade this problem in a container identical to the one we provided to you.

Note on submission: For Parts B through E in this problem, please submit the commands in a text file called P3.sh. Each line should correspond to a command representing the solution in order (i.e. line 1 for Part B, line 2 for Part C, etc.). Part E has two parts that should be put on different lines. Hence your file should be 5 lines and look something like:

#!/bin/bash
solution to part B
solution to part C
solution to part D
solution to first part of E
solution to second part of E

Of course, it will look different because we have witheld the solutions.

Part F should be submitted in a file called P3F.sh.

Example Question: Use two Unix commands (on a single line) to count the number of words in a file.
Example Solution:

Part A: Setting up and using a Docker container.¶

Due to the varying outputs across differing operating systems, we strongly recommend that you use the following commands to run and test your commands on our Linux environment.

1. Install Docker on your machine from https://docs.docker.com/get-docker/.
2. Make sure the Docker application is running.
3. Open a command line terminal.
4. Make a pull request to our Docker container (docker pull pargaw/cs107_hw1)
5. Run the Docker container (docker run -it pargaw/cs107_hw1)
6. You should see that your command line prompt changes to root@..... You're now in the Docker container, which is running Linux!
7. You can now test Unix commands on here for your homework!!
8. Type exit in the Docker container window to leave the container.

Note: You will learn more about virtual environments, machines and containers in a future lecture.

Part B [5 pts]:¶

You are provided with apollo13.txt in the Docker container. Write a command that writes the number of lines in the file that contain a numerical digit to an output file called apollo_out.txt. Your command should assume that the text file is in the current directory.

For example, if the provided text file contained:

a9
2c
abd
939

Then the resulting command should output a file with $3$.

Part C [5 pts]:¶

Write a command using grep that ouputs a single line that tells what the --count or -c option does in grep.

Hints:

• When using Linux, you might need the results of grep --help.
• You can search for either --count or '-c'.
• Your solution must return a single line.

For example, the output of the command for the -V option in grep would be:

-V, --version display version information and exit.

Part D [5 pts]: Write a command that counts the total number of .py files in your current directory.¶

For example, if the current directory contains the following:
p1.py
Documents/
checkGrades.c
rejected_hard_problem.py

Then the output of the command should be $2$.

Note: We did not provide any .py files in the Docker container. If you want to play around, feel free to create some artifical .py files to test things. An easy to to create a test file is to use the touch command: touch test1.py will create an empty file called test1.py.

Part E [5 pts]:¶

Write a command that counts the number of:

1. Files in the current directory and sub-directories that "others" DO NOT have read and write access to.
2. Files and directories in the current directory that "others" DO NOT have read and write access to.

For example, if the current directory contains the following:
File1 ← read/write for others
File2 ← read/write for users
Subdirectory1/ ← read/write for others
  File3 ← read/write for others
Subdirectory2/ ← read/write for users
  File4 ← read/write for users
  File5 ← read/write for users

Then the output of the command for part 1 should be $3$, and the output for part 2 should be $2$.

Part F [10 pts]:¶

Write a bash script that counts and prints the number of lines of each file in the current directory along with its name. Your resulting file should be called P3F.sh.

For example, if the directory contains the following:
p1.py
Documents/
checkGrades.c
rejected_hard_problem.py

Then the output will look like:
p1.py 10
checkGrades.c 100
rejected_hard_problem.py 10000.

Final Deliverables for Problem 3¶

At this point, we expect to find the following files in your HW1-dev branch in the HW1-final/ subdirectory:

• P3.sh

  • The following lines should be in the file (in this exact order):

      Command for Part B
      Command for Part C
      Command for Part D
      Command 1 for Part E
      Command 2 for Part E

• P3F.sh

Homework Session: Check-In¶

Using command line on different systems

Note. This question is optional and will not be graded, though it is important to understand when using the command line. This topic will be discussed in more detail in your homework groups.

• Did you try running your commands on your local machines?
  • If so, did you notice different answers or errors when running the same commands?
• What are some ways to combat this?

Problem 4 [10pts]: Write some Markdown¶

The task in this problem is to re-write the README.md document in the homework/ directory. This will be done with the Markdown language.

Markdown is a very easy language to work with. Here's a Markdown cheatsheet with everything you need to know: Markdown Cheatsheet.

The task is to write a short Markdown document. Your Markdown document must have at least one of the following:

• Two different types of headers
• A List
• A table
• A link

You can include other features if you want.

Final Deliverables for Problem 4¶

• An updated README.md in the homework/ directory

Problem 5: Git crash course¶

A vital part of software development is being able to manage changes and the entire workflow of a project. Version Control Systems (VCS) are the primary tools used to keep track of actual changes to code. Git is the most common VCS used and it is what we will use for this course.

Grading on this problem will be based on the edits you make to your branch as well as a screenshot that we'll ask for during the assignment.

Part A [10 pts]: Getting Started¶

Before we make changes, we're first going to start working off an existing git repository (repo). Go to https://github.com/cs107-sys-dev/playground. There will be a green button that says Code, Clone or Download. The website will give you a URL.

Copy that URL. Open a command line terminal. Go to a directory outside the CS107 repo (remember to use Unix commands like cd).

In the command line type: git clone *URL_COPIED*

IMPORTANT: Please clone the playground repo outside of your CS107 repo. At a minimum, you should have a directory called Classes/ with subdirectories for each class you're taking this semester. One of these directories should be CS107 (or AC207 or CSCI-E-207). Inside CS107/ you should have your course repo from Problem 2 above. And now you should clone this little playground repo into your CS107/ directory. Again, it should not be inside your course repo.

Now you'll see an entirely set up git repo for you with all the content.

The problem with this approach is that you do not have push access to this public repo. Therefore, when you try to do the next exercises and push your changes, you will get a permission denied error. There are two ways to solve this:

1. Have the repository owner give you push access --- This will be challenging with a course this size.
2. Create your own fork of the repository and make a pull request (PR) back to the original repository when you've made all the changes. This is the approach that you will take in this assignment. It's an excellent skill to acquire.

To use the forking workflow, use the following steps:

1. Delete the playground repo that you just cloned. You're going to be working off of a fork named the same thing.
2. Fork the repo by clicking the fork button in the upper right hand corner
3. Now you have a forked copy of the repository! From this forked repo, copy the repo link (as discussed above) and type git clone *URL_COPIED* into your command line. Now you have a local copy of the forked repository, in which you have complete control of the project and can make whatever changes you want.
4. Complete Parts B and C on your fork.

Part B [25 pts]: Branches and Merge Conflicts¶

One of the most fundamental concepts in git is the concept of a branch.

A branch is an area from which code can be updated or modified and continually branched. It's good practice to NEVER make changes directly on master but create a branch to work off of during development. To see a list of branches type

git branch -a

If you've started this assignment early, you may only see a few branches. If you started after others you may see many other branches! To switch to the TF branch type git checkout TFtestbranch

git checkout moves you from one branch to another.

If you type ls you'll see that there are different files on this branch. Different branches can contain different files. They can also contain different content in exisiting files.

Next you will make your own branch. First switch back to master - type git checkout master.

Now you're ready to create your first branch (you should name it firstname_lastname) git branch *firstname_lastname*.

To move to this branch use git checkout *firstname_lastname*.

And ta-da you've created your own branch!

Create one more branch. You can use a single command git checkout -b *firstname_lastname_changes*. This is simply doing a combination of branch and checkout in one line.

Now update the README.md. Open up README.md and add your name to the top of the file just under the header. Use Markdown format. Save and close the file. Note: You may use your text editor of choice to edit the file. DO NOT use Word or Google Docs.

In the same terminal, type git status. This is how you see changes to the repo.

git status will summarize everything that is going on in your branch and is immensely useful (running git status is always a good thing to do). A good practice is to use git status after every git action you take.

Now we'll actually add the file. Type git add README.md.

This tells git that you are preparing to commit README.md (formally this is called staging). Now do git status again and you should see something like:

Now type git commit.

You'll see a texteditor pop up. Don't get worried by the stuff already there - on the first line add a description of what this commit is doing (i.e updating the README with your name). Note: Depending on your system, git commit may open different kinds of text editors. In general, the default text editor is vim (or vi). You can change the default if you prefer. For now, if you're not familiar with vim, simply enter the following commands:

• Type the letter i for insert. This opens insert mode.
• Enter your message.
• Press the escape button, esc. This exits insert mode.
• Type :wq. The : enters command mode. w means write and q means quit.

Congrats! You've made your first commit.

(You can also do git commit -m "YOUR COMMIT MESSAGE" to avoid the texteditor popping up.)

If you type git status you'll see git telling you there's nothing new to commit.

However, this has only changed your history locally! You still need to let the world (that is remote/github) know that you've made these changes. To accomplish this, type git push in the terminal.

But when you do that you'll get a funny error message

fatal: The current branch *BRANCH NAME* has no upstream branch

It'll also tell you how to resolve the issue, by running:

git push --set-upstream origin *firstname_lastname_changes*

What this is doing is it's telling git which remote repo it needs to push to. By default if you type in "origin" it will reference the remote repo that you clone/pulled from originally (each time you add a new branch you'll have to set the upstream if you want to push it to the remote).

Follow those instructions and then push. (If you haven't set up authentication you'll have to type in your github username/password).

After that it'll be updated in remote (Github). Your collaboraters (and your TFs if you gave them access) will now be able to pull from your branch!

Now let's try something else. Let's go back to your branch firstname_lastname: git checkout *firstname_lastname*.

If you open up the README - you'll see that your name isn't on the README. This is expected as you only made changes on a different branch.

Now we'll update this README - instead of your name add some of the courses you're planning on taking this semester. Make these changes on the same line that you did previously. Now commit (don't push!) like above.

Now we'd like our current branch to reflect the changes from the branch we made earlier (branch firstname_lastname_changes). To combine branches while on branch firstname_lastname type git merge *firstname_lastname_changes*.

You should notice in your terminal AUTOMATIC merge failed; fix conflicts and then commit.

Don't worry - this is expected! What has happened is the history of these two branches have diverged and git doesn't know what the result should be (should it keep your changes or your colleagues changes? which one is correct?). In some cases (if the changes are distinct enough) git is smart enough to do the merge. However, in this case it cannot and there is a merge conflict. Git keeps both changes and provides annotations to help you decide which one to keep. We'll walk you through resolving the merge conflict in this exercise.

Type git status - it should tell you which files neeed to be merged.

Lets open up the README.

Take a screenshot of the README and call it P5_merge.png (you don't need add the red text). Commit/push this file to your firstname_lastname branch.

You need to go in and manually update it to reflect the final state you want (and then remove the <<<<<, =====, >>>>> symbols).

You can see that in git status there are unmerged paths.

Once you're ready you should have a clean file to commit. Go ahead and add the file and commit with a message that says you've resolved the merge conflict. You may see an automatic commit message. You can keep that one or write your own. Up to you, but informative messages are always better than generic ones.

Push to remote and congrats for solving the merge conflicts!

Connecting back to lecture
Recall the meaning of the symbols:

• The content between <<<<<<< HEAD and ======= shows what the current state of the HEAD is. You can think of this as what existed before you tried to merge your changes. This is the "head" of the current branch; the most recent change.
• The content between ======= and >>>>>>> xxxxxxx shows what changes are trying to be merged in. This is essentially the conflict. The xxxxxxx will usually be the conflicting branch name of the conflicting commit hash.
• Your job is to decide which thing to keep. In practice, you might need to call up your colleagues to discuss. In the current exercise you can just do this yourself because you know which changes to keep.

Don't be afraid of merge conflicts!

Let's add one more feature.

In your firstname_lastname branch add another line that says what year you are (G1, junior, whatever it is). Follow the usual add - commit - push cycle to get the changes to the branch on the remote repo.

Now suppose you want to revert this change (or any change really). The magic of version control is that it's relatively straight forward to go back to a previous version!

To revert a commit - first identify the commit you want to rollback. To do this type git log

You'll see several commits - the commit (hash id) is the long string (in this example my latest commit starts with a5a......). You can get out of the log by just typing q.

Here you want to revert your last commit. To accomplish this, just type git revert *hash_id*.

All this does is add a new commit, which is just restoring to the state that occured before the above commit. Type git status to see exactly what's going on. And do git log again to see the new commit!

To push the change simply git push. Now everyone can see the most up-to-date version of this branch which will be your version of the file with just your name and courses.

Note that we didn't "delete" the offending commit. Instead we added a new commit on top of the offending commit - that way there's still a history of all the changes that occured. git revert is history preserving in that the history stored in git is never modified. This is a very good thing.

You may also want to revert changes that you've committed locally but haven't push to remote. This is what git reset will do.

Recap and Deliverables¶

• At this point, you have learned how to create branches and use them for development, how to resolve merge conflicts, and how to revert work from your branch. These will be helpful going forward for your class project and future coding projects.
• By the end of this part, you should expect to have an updated README.md file and the P5_merge.png screenshot on your firstname_lastname branch in your playground/ repo.

Part C [15 pt]: Forking and setting up remote¶

In the previous section we were mostly working off of branches. Branches are great but sometimes we need to be able to fully change a given project. Often times a project might be spun off an existing repo and take on a life of its own (e.g. Ubuntu is a fork of the Linux project). The fork may still contribute back to the original project from time to time, but it is usually meant to be an entirely new project. Branches on the other hand are meant to be confined within a single repo.

We've already forked the original playground repo in Part A and cloned that fork to your local machine. Sometimes even with a forked version we would like to track another project (for example the original repo).

To check the status of the remote, type git remote -v

You'll see two lines that both point to your current forked version.

To set an upstream version you can type git remote add upstream https://github.com/cs107-sys-dev/playground.git. Note that this URL simply came from the Github site of the cs107-sys-dev/playground repo.

Now if you type git remote -v you'll see that upstream is set to the original repo.

Take a screenshot of this file and name it P5_remote.png. Commit and push this file to your firstname_lastname branch.

The diagram you've created is essentially this (poorly) done cartoon:

You have two remote repos from which you can fetch (receive changes from remote) and push (make changes to remote). There are often situations where you might fetch from one remote (say a fully tested and reflecting the latest good build) and push to another (a quality assessment remote). You can specify which remote you wish to pull/push from (e.g. git pull origin master vs. git pull upstream master).

Final Deliverables for Problem 5¶

That was a lot of stuff! At this point, working from your firstname_lastname branch, issue a pull request back to the original repository. A member of the teaching staff will have a look at your PR. Your branch should include the following:

• an updated README.md
• P5_merge.png
• P5_remote.png

We will also be able to see your commit history to make sure you followed all the steps.

Some Unix References and Examples¶

This extra set of exercises is for your reference in case you don't have much experience with Unix. Please do your best to become comfortable with the command line throughout the semester. There are plenty of resources available on learning how to use the command line efficiently. Here is one to get you started: http://linuxcommand.org/lc3_learning_the_shell.php#contents.

Macs come with a command line app (terminal.app) and since Macs are loosely similar to Unix, the terminal commands are pretty much the same as those in Linux distributions. Windows is a bit different, and because of this, I had you install git BASH if you are using a Windows machine. What this means is that, even if you are using a Windows OS, you can still use the nice Unix commands as long as you are in the git BASH terminal.

It is true that there are various options for using git from the Desktop (e.g. see GitHub Desktop). However, it is more useful for you to be able to do things from the command line. The reasons for this are that the command line is not going anywhere any time soon and that I can't predict what GUI your future company/group will want you to use. You'll be much more versatile using the command line.

What this means is that you need to learn a little bit about how to navigate with the command line. There are a few essential commands that you absolutely must know. You'll pick up the rest as you go through your life. Here are the essential ones:

• ls: list the contents of the current directory
• cd: change to a new directory
• mv: rename a directory or file OR move a directory or file to a new location
• cp: copy a directory or file
• pwd: print the working directory
• mkdir: create a new directory
• rm: remove a file (you can remove a directory with rm -r)

Let's try to work with these a bit. In the following steps, you should hit the return key after typing each command.

1. Open up a terminal and type pwd. The output should be your home directory.
2. Next type ls. You should see all the files and directories in your home directory. Notice that if you type ls -l you get a list with more information where the files and directories are ordered alphabetically. Typing ls -lt sorts the list by date/time created with the most recent file/directory at the top. ls -ltr just reverses the order of the sort; now the most recent file/directory is at the bottom.
3. Now type mkdir mydir. Voila! You have just created a new directory. Try ls again. Do you see the new directory?
4. To enter that directory type cd mydir. You just changed into your new directory. Type pwd to see where you are.
5. Now that you're inside your new directory, let's create a file in there.
   • Type echo 'Hello world!' > newfile.txt
   • Type ls. You should see the next text file!
   • To see the contents of the file, type cat newfile.txt.
6. Suppose you now want to rename newfile.txt to README.txt. Just execute mv newfile.txt README.txt. Type ls again if you wish.
7. Maybe you really wanted to keep a copy of newfile.txt. Just do cp README.txt newfile.txt. As usual, ls will show you what you want to see.
8. What if newfile.txt should actually be in a different directory? Let's do the following:
   • mkdir newdir
   • mv newfile.txt newdir/
   • Altneratively, you could give the entire path of the new directory (not necessary here). Here's how you would do that:
     • pwd (just to see the current path)
     • mv newfile.txt path_from_pwd/newdir/
9. That was fun, but this directory is completely useless to use now. Let's delete it.
   • First, you need to get out of the current directory. You can't delete a directory from within it!
   • To go up one directory, just type cd ...
     • Unix Note: In Unix, . stands for the current directory and .. stands for the parent directory (that is, the directory containing the current directory. Hence, cd ../ changes to the parent directory of the current directory.
     • Unix Note: If you want to go up two directories, just do cd ../../. You can use the same pattern for $n$ directories.
   • Now type rm mydir. You should see the message rm: mydir/: is a directory. rm cannot be used to remove directories as-is. It can only remove files.
   • To remove the directory type rm -rv mydir. The -r option says to recursively remove the directory and any contents. The -v option says to be verbose while removing files. You don't really need the -v option, but I like use it so I can see progress.

That was a whirlwind tour through a few of the more useful Unix commands. It won't take you long to use them effectively, but you must use them.

Note: If you want more information about any command, just type man command_name into the terminal. This will bring up the manual page for that command. You will see all sorts of information. To scroll down, just hit the space bar. To exit the man page just type q (for quit). Here's an example of a portion of the man page for the rm command:

Bonus Command¶

Suppose you want to copy a file from a different location to your current directory. Well, recall that the current directory is represented by the . object. Here are a few things you can do:

• cp path_to_file/file .: Will copy file to your current location
• cp path_to_file/file ./sub_dir: Will copy file to the directory sub_dir in your current location
• cp ../file .: Will copy file from the parent directory to your current location There are other, similar patterns to accomplish roughly the same thing. You'll ../figure them out as you go through life.