INSY 3400
Programming Assignment 1
It is not always practical to compute the steady state or absorbing probabilities analytically for
complex systems. In this assignment, you will use the programming language of your choosing
to compute the specified probabilities. It is recommended that you use python, but if you have
not yet taken INSY 3010 you may prefer to use matlab or another language that you have used
before.
Consider the skittle and m&m problem from the homework (and exam), that is restated below.
Three mini bags of skittles and three mini bags of M&Ms are distributed in two candy
bowls (one red, one brown) in such a way that each bowl contains three bags of candy. At
each step we draw one candy bag from each dish and exchange them. Let Xn be the
number of bags of skittles in the red candy bowl at time n.
The recommended steps to solve for the steady state probabilities empirically are listed below.
• Create a list of lists for the one-step transition probability matrix P using the appropriate
values. Keep in mind that this is a problem you have already worked, so you may refer
back to your homework to obtain P.
• Generate a random starting state, s, from {0,1,2,3} and record it in the list of states visited
• Create list, referred to as lim below, populated with 0s, to store the number of times each
state has been visited.
• Execute a transition 10,000 times:
o Generate a random uniform number between 0 and 1.
o Depending on what the current state is, determine what the next state will be by
using the conditional pmf for the given state.
Remember that you can convert the conditional pmf to a cdf, and then find
the bracket containing the generated random number
For the row of the transition matrix that corresponds to the current state,
add the transition probabilities one by one from left to right until the sum
exceeds the value of the random number. The first time when that happens
determines the next state.
o Add 1 to the entry in lim corresponding to the selected state.
• Divide each entry in lim by the total number of transitions executed to obtain the
estimated steady state probabilities.
Submission Instructions
Upload a code file with all of your code and a pdf with your submission report.
For your code file:
• Put your name in the comments at the top of the file
• Include your empirically determined steady state probabilities in the comments at the top
of the file
• Your code must be commented
• If you are using python, submit a single py file
• If you are not using python, submit all of the files needed to execute your experiment.
For the report:
• The header should include your name (as it appears on canvas) and the name of the
assignment.
• Write a one paragraph introduction that outlines the problem, objective, and states which
programming language you are choosing to use. If you are not using python, include
brief instructions for how to use your submitted files to re-create your results.
• Write a one paragraph analysis of the results. Clearly state what your randomly selected
initial state is, and what computed steady-state values are. Discuss any similarities and/or
differences to the steady state values you have computed by hand in previous
assignments.
Programming Assignment 1
It is not always practical to compute the steady state or absorbing probabilities analytically for
complex systems. In this assignment, you will use the programming language of your choosing
to compute the specified probabilities. It is recommended that you use python, but if you have
not yet taken INSY 3010 you may prefer to use matlab or another language that you have used
before.
Consider the skittle and m&m problem from the homework (and exam), that is restated below.
Three mini bags of skittles and three mini bags of M&Ms are distributed in two candy
bowls (one red, one brown) in such a way that each bowl contains three bags of candy. At
each step we draw one candy bag from each dish and exchange them. Let Xn be the
number of bags of skittles in the red candy bowl at time n.
The recommended steps to solve for the steady state probabilities empirically are listed below.
• Create a list of lists for the one-step transition probability matrix P using the appropriate
values. Keep in mind that this is a problem you have already worked, so you may refer
back to your homework to obtain P.
• Generate a random starting state, s, from {0,1,2,3} and record it in the list of states visited
• Create list, referred to as lim below, populated with 0s, to store the number of times each
state has been visited.
• Execute a transition 10,000 times:
o Generate a random uniform number between 0 and 1.
o Depending on what the current state is, determine what the next state will be by
using the conditional pmf for the given state.
Remember that you can convert the conditional pmf to a cdf, and then find
the bracket containing the generated random number
For the row of the transition matrix that corresponds to the current state,
add the transition probabilities one by one from left to right until the sum
exceeds the value of the random number. The first time when that happens
determines the next state.
o Add 1 to the entry in lim corresponding to the selected state.
• Divide each entry in lim by the total number of transitions executed to obtain the
estimated steady state probabilities.
Submission Instructions
Upload a code file with all of your code and a pdf with your submission report.
For your code file:
• Put your name in the comments at the top of the file
• Include your empirically determined steady state probabilities in the comments at the top
of the file
• Your code must be commented
• If you are using python, submit a single py file
• If you are not using python, submit all of the files needed to execute your experiment.
For the report:
• The header should include your name (as it appears on canvas) and the name of the
assignment.
• Write a one paragraph introduction that outlines the problem, objective, and states which
programming language you are choosing to use. If you are not using python, include
brief instructions for how to use your submitted files to re-create your results.
• Write a one paragraph analysis of the results. Clearly state what your randomly selected
initial state is, and what computed steady-state values are. Discuss any similarities and/or
differences to the steady state values you have computed by hand in previous
assignments.
