\section{Probabilistic Bits}


\definition{}

As we already know, a \textit{classical bit} may take the values \texttt{0} and \texttt{1}. \par
We can model this with a two-sided coin, one face of which is labeled \texttt{0}, and the other, \texttt{1}. \par

\vspace{2mm}

Of course, if we toss such a \say{bit-coin,} we'll get either \texttt{0} or \texttt{1}. \par
We'll denote the probability of getting \texttt{0} as $p_0$, and the probability of getting \texttt{1} as $p_1$. \par
As with all probabilities, $p_0 + p_1$ must be equal to 1.


\vfill


\definition{}

Say we toss a \say{bit-coin} and don't observe the result. We now have a \textit{probabilistic bit}, with a probability $p_0$
of being \texttt{0}, and a probability $p_1$ of being \texttt{1}.

\vspace{2mm}

We'll represent this probabilistic bit's \textit{state} as a vector:
$\left[\begin{smallmatrix}
	p_0 \\ p_1
\end{smallmatrix}\right]$ \par
We do \textbf{not} assume this coin is fair, and thus $p_0$ might not equal $p_1$.

\note{
	This may seem a bit redundant: since $p_0 + p_1$, we can always calculate one probability given the other. \\
	We'll still include both probabilities in the state vector, since this provides a clearer analogy to quantum bits.
}


\vfill


\definition{}

The simplest probabilistic bit states are of course $[0]$ and $[1]$, defined as follows:
\begin{itemize}
	\item $[0] = \left[\begin{smallmatrix} 1 \\ 0 \end{smallmatrix}\right]$
	\item $[1] = \left[\begin{smallmatrix} 0 \\ 1 \end{smallmatrix}\right]$
\end{itemize}
That is, $[0]$ represents a bit that we known to be \texttt{0}, \par
and $[1]$ represents a bit we know to be \texttt{1}.

\vfill


\definition{}
$[0]$ and $[1]$ form a \textit{basis} for all possible probabilistic bit states: \par
Every other probabilistic bit can be written as a \textit{linear combination} of $[0]$ and $[1]$:

\begin{equation*}
	\begin{bmatrix} p_0 \\ p_1 \end{bmatrix}
	=
	p_0 \begin{bmatrix} 1 \\ 0 \end{bmatrix} +
	p_1 \begin{bmatrix} 0 \\ 1 \end{bmatrix}
	=
	p_0 [0] + p_1 [1]
\end{equation*}


\vfill
\pagebreak

\problem{}
Every possible state of a probabilistic bit is a two-dimensional vector. \par
Draw all possible states on the axis below.


\begin{center}
	\begin{tikzpicture}[scale = 2.0]
		\fill[color = black] (0, 0) circle[radius=0.05];
		\node[below left] at (0, 0) {$\left[\begin{smallmatrix} 0 \\ 0 \end{smallmatrix}\right]$};

		\draw[->] (0, 0) -- (1.2, 0);
		\node[right] at (1.2, 0) {$p_0$};
		\fill[color = oblue] (1, 0) circle[radius=0.05];
		\node[below] at (1, 0) {$[0]$};

		\draw[->] (0, 0) -- (0, 1.2);
		\node[above] at (0, 1.2) {$p_1$};
		\fill[color = oblue] (0, 1) circle[radius=0.05];
		\node[left] at (0, 1) {$[1]$};
	\end{tikzpicture}
\end{center}

\begin{solution}
	\begin{center}
		\begin{tikzpicture}[scale = 2.0]
			\fill[color = black] (0, 0) circle[radius=0.05];
			\node[below left] at (0, 0) {$\left[\begin{smallmatrix} 0 \\ 0 \end{smallmatrix}\right]$};

			\draw[ored, -, line width = 2] (0, 1) -- (1, 0);


			\draw[->] (0, 0) -- (1.2, 0);
			\node[right] at (1.2, 0) {$p_0$};
			\fill[color = oblue] (1, 0) circle[radius=0.05];
			\node[below] at (1, 0) {$[0]$};

			\draw[->] (0, 0) -- (0, 1.2);
			\node[above] at (0, 1.2) {$p_1$};
			\fill[color = oblue] (0, 1) circle[radius=0.05];
			\node[left] at (0, 1) {$[1]$};
		\end{tikzpicture}
	\end{center}
\end{solution}


\vfill
\pagebreak





\section{Measuring Probabilistic Bits}


\definition{}
As we noted before, a probabilistic bit represents a coin we've tossed but haven't looked at. \par
We do not know whether the bit is \texttt{0} or \texttt{1}, but we do know the probability of both of these outcomes. \par

\vspace{2mm}

If we \textit{measure} (or \textit{observe}) a probabilistic bit, we see either \texttt{0} or \texttt{1}---and thus our
knowledge of its state is updated to either $[0]$ or $[1]$, since we now certainly know what face the coin landed on.

\vspace{2mm}

Since measurement changes what we know about a probabilistic bit, it changes the probabilistic bit's state.
When we measure a bit, it's state \textit{collapses} to either $[0]$ or $[1]$, and the original state of the
bit vanishes. We \textit{cannot} recover the state $[x_0, x_1]$ from a measured probabilistic bit.


\definition{Multiple bits}
Say we have two probabilistic bits, $x$ and $y$, \par
with states
$[x]=[ x_0, x_1]$
and
$[y]=[y_0, y_1]$

\vspace{2mm}

The \textit{compound state} of $[x]$ and $[y]$ is exactly what it sounds like: \par
It is the probabilistic two-bit state $\ket{xy}$, where the probabilities of the first bit are
determined by $[x]$, and the probabilities of the second are determined by $[y]$.




\problem{}<firstcompoundstate>
Say $[x] = [\nicefrac{2}{3}, \nicefrac{1}{3}]$ and $[y] = [\nicefrac{3}{4}, \nicefrac{1}{4}]$. \par
\begin{itemize}[itemsep = 1mm]
	\item If we measure $x$ and $y$ simultaneously, \par
	what is the probability of getting each of \texttt{00}, \texttt{01}, \texttt{10}, and \texttt{11}?


	\item If we measure $y$ first and observe \texttt{1}, \par
	what is the probability of getting each of \texttt{00}, \texttt{01}, \texttt{10}, and \texttt{11}?
\end{itemize}
\note[Note]{$[x]$ and $[y]$ are column vectors, but I've written them horizontally to save space.}


\vfill





\problem{}
Say $[x] = [\nicefrac{2}{3}, \nicefrac{1}{3}]$ and $[y] = [\nicefrac{3}{4}, \nicefrac{1}{4}]$. \par
What is the probability that $x$ and $y$ produce different outcomes?

\vfill
\pagebreak











\section{Tensor Products}

\definition{Tensor Products}
The \textit{tensor product} of two vectors is defined as follows:
\begin{equation*}
	\begin{bmatrix}
		x_1 \\ x_2
	\end{bmatrix}
	\otimes
	\begin{bmatrix}
		y_1 \\ y_2
	\end{bmatrix}
=
	\begin{bmatrix}
		x_1
		\begin{bmatrix}
			y_1 \\ y_2
		\end{bmatrix}

		\\[4mm]

		x_2
		\begin{bmatrix}
			y_1 \\ y_2
		\end{bmatrix}
	\end{bmatrix}
=
	\begin{bmatrix}
		x_1y_1 \\[1mm]
		x_1y_2 \\[1mm]
		x_2y_1 \\[1mm]
		x_2y_2 \\[0.5mm]
	\end{bmatrix}
\end{equation*}


That is, we take our first vector, multiply the second
vector by each of its components, and stack the result.
You could think of this as a generalization of scalar
mulitiplication, where scalar mulitiplication is a
tensor product with a vector in $\mathbb{R}^1$:
\begin{equation*}
	a
	\begin{bmatrix}
		x_1 \\ x_2
	\end{bmatrix}
=
	\begin{bmatrix}
		a_1
	\end{bmatrix}
	\otimes
	\begin{bmatrix}
		y_1 \\ y_2
	\end{bmatrix}
=
	\begin{bmatrix}
		a_1
		\begin{bmatrix}
			y_1 \\ y_2
		\end{bmatrix}
	\end{bmatrix}
=
	\begin{bmatrix}
		a_1y_1 \\[1mm]
		a_1y_2
	\end{bmatrix}
\end{equation*}

\problem{}
Say $x \in \mathbb{R}^n$ and $y \in \mathbb{R}^m$. \par
What is the dimension of $x \otimes y$?

\vfill

\problem{}<basistp>
What is the pairwise tensor product
$
\Bigl\{
	\left[
	\begin{smallmatrix}
		1 \\ 0 \\ 0
	\end{smallmatrix}
	\right],
	\left[
	\begin{smallmatrix}
		0 \\ 1 \\ 0
	\end{smallmatrix}
	\right],
	\left[
	\begin{smallmatrix}
		0 \\ 0 \\ 1
	\end{smallmatrix}
	\right]
\Bigr\}
\otimes
\Bigl\{
	\left[
	\begin{smallmatrix}
		1 \\ 0
	\end{smallmatrix}
	\right],
	\left[
	\begin{smallmatrix}
		0 \\ 1
	\end{smallmatrix}
	\right]
\Bigr\}
$?

\note{in other words, distribute the tensor product between every pair of vectors.}

\vfill










\problem{}
What is the \textit{span} of the vectors we found in \ref{basistp}? \par
In other words, what is the set of vectors that can be written as linear combinations of the vectors above?

\vfill

% This is wrong, but there's something here.
% maybe fix later?
%
%Look through the above problems and convince yourself of the following fact: \par
%If $a$ is a basis of $A$ and $b$ is a basis of $B$, $a \otimes b$ is a basis of $A \times B$. \par
%\note{If you don't understand what this says, ask an instructor. \\ This is the reason we did the last few problems!}
%
%\begin{instructornote}
%	\textbf{The idea here is as follows:}
%
%	If $a$ is in $\{\texttt{0}, \texttt{1}\}$ and $b$ is in $\{\texttt{0}, \texttt{1}\}$,
%	the values $ab$ can take are
%	$\{\texttt{0}, \texttt{1}\} \times \{\texttt{0}, \texttt{1}\} = \{\texttt{00}, \texttt{01}, \texttt{10}, \texttt{11}\}$.
%
%	\vspace{2mm}
%
%	The same is true of any other state set: if $a$ takes values in $A$ and $b$ takes values in $B$, \par
%	the compound state $(a,b)$ takes values in $A \times B$.
%
%	\vspace{2mm}
%
%	We would like to do the same with probabilistic bits. \par
%	Given bits $\ket{a}$ and $\ket{b}$, how should we represent the state of $\ket{ab}$?
%\end{instructornote}

\pagebreak






\problem{}
Say $[x] = [\nicefrac{2}{3}, \nicefrac{1}{3}]$ and $[y] = [\nicefrac{3}{4}, \nicefrac{1}{4}]$. \par
What is $[x] \otimes [y]$? How does this relate to \ref{firstcompoundstate}?

\vfill



\problem{}
The compound state of two vector-form bits is their tensor product. \par
Compute the following. Is the result what we'd expect?
\begin{itemize}
	\item $[0] \otimes [0]$
	\item $[0] \otimes [1]$
	\item $[1] \otimes [0]$
	\item $[1] \otimes [1]$
\end{itemize}
\hint{
	Remember that
	$[0] = \left[\begin{smallmatrix} 1 \\ 0 \end{smallmatrix}\right]$
	and
	$[1] = \left[\begin{smallmatrix} 0 \\ 1 \end{smallmatrix}\right]$.
}


\vfill









\problem{}<fivequant>
Of course, writing $[0] \otimes [1]$ is a bit excessive. We'll shorten this notation to $[01]$. \par

\vspace{2mm}

In fact, we could go further: if we wanted to write the set of bits $[1] \otimes [1] \otimes [0] \otimes [1]$, \par
we could write $[1101]$---but a shorter alternative is $[13]$, since $13$ is \texttt{1101} in binary.

\vspace{2mm}

Write $[5]$ as three-bit probabilistic state. \par

\begin{solution}
	$[5] = [101] = [1] \otimes [0] \otimes [1] = [0,0,0,0,0,1,0,0]^T$ \par
	Notice how we're counting from the top, with $[000] = [1,0,...,0]$ and $[111] = [0, ..., 0, 1]$.
\end{solution}

\vfill






\problem{}
Write the three-bit states $[0]$ through $[7]$ as column vectors. \par
\hint{You do not need to compute every tensor product. Do a few and find the pattern.}






\vfill
\pagebreak

















\section{Operations on Probabilistic Bits}

Now that we can write probabilistic bits as vectors, we can represent operations on these bits
with linear transformations---in other words, as matrices.

\definition{}
Consider the NOT gate, which operates as follows: \par
\begin{itemize}
	\item $\text{NOT}[0] = [1]$
	\item $\text{NOT}[1] = [0]$
\end{itemize}
What should NOT do to a probabilistic bit $[x_0, x_1]$? \par
If we return to our coin analogy, we can think of the NOT operation as
flipping a coin we have already tossed, without looking at its state.
Thus,
\begin{equation*}
	\text{NOT} \begin{bmatrix}
		x_0 \\ x_1
	\end{bmatrix} = \begin{bmatrix}
		x_1 \\ x_0
	\end{bmatrix}
\end{equation*}


\begin{ORMCbox}{Review: Multiplying Vectors by Matrices}{black!10!white}{black!65!white}
	\begin{equation*}
		Av =
		\begin{bmatrix}
			1 & 2 \\
			3 & 4 \\
		\end{bmatrix}
		\begin{bmatrix}
			v_0 \\ v_1
		\end{bmatrix}
		=
		\begin{bmatrix}
			1v_0 + 2v_1 \\
			3v_0 + 4v_1
		\end{bmatrix}
	\end{equation*}

	Note that each element of $Av$ is the dot product of a row in $A$ and a column in $v$.
\end{ORMCbox}

\problem{}
Compute the following product:
\begin{equation*}
	\begin{bmatrix}
		1 & 0.5 \\ 0 & 1
	\end{bmatrix}
	\begin{bmatrix}
		3 \\ 2
	\end{bmatrix}
\end{equation*}


\vfill

\generic{Remark:}
Also, recall that every matrix is linear map, and that every linear map may be written as a matrix. \par
We often use the terms \textit{matrix}, \textit{transformation}, and \textit{linear map} interchangably.

\pagebreak


\problem{}
Find the matrix that represents the NOT operation on one probabilistic bit.

\begin{solution}
	\begin{equation*}
		\begin{bmatrix}
			0 & 1 \\ 1 & 0
		\end{bmatrix}
	\end{equation*}
\end{solution}

\vfill


\problem{Extension by linearity}
Say we have an arbitrary operation $M$. \par
If we know how $M$ acts on $[1]$ and $[0]$, can we compute $M[x]$ for an arbitrary state $[x]$? \par
Say $[x] = [x_0, x_1]$.
\begin{itemize}
	\item What is the probability we observe $0$ when we measure $x$?
	\item What is the probability that we observe $M[0]$ when we measure $Mx$?
\end{itemize}

\vfill

\problem{}<linearextension>
Write $M[x_0, x_1]$ in terms of $M[0]$, $M[1]$, $x_0$, and $x_1$.


\begin{solution}
	\begin{equation*}
		M \begin{bmatrix}
			x_0 \\ x_1
		\end{bmatrix}
		=
		x_0 M \begin{bmatrix}
			1 \\ 0
		\end{bmatrix}
		+
		x_1 M \begin{bmatrix}
			0 \\ 1
		\end{bmatrix}
		=
		x_0 M [0] +
		x_1 M [1]
	\end{equation*}
\end{solution}



\vfill

\generic{Remark:}
Every matrix represents a \textit{linear} map, so the following is always true:
\begin{equation*}
	A \times (px + qy) = pAx + qAy
\end{equation*}
\ref{linearextension} is just a special case of this fact.

\pagebreak