\section{Introduction}
% Detailed summary of the problem.
% Discuss why addressing this problem is important.


\textit{Celeste} \cite{celestegame} is a fairly successful 2018 platformer, known for high-quality level design, a vibrant speedrunning\footnotemark{} community, and brutally difficult progression. It is based on \textit{Celeste Classic}, a 4-day game jam project by the same authors. There are a few reasons we chose to create an agent for \textit{Celeste Classic}:

\footnotetext{\textit{speedrunning:} a competition where participants try to complete a game as quickly as possible, often abusing bugs and design mistakes.}

\vspace{4mm}

\noindent
\begin{minipage}{0.5\textwidth}
	\noindent
	1: \textit{Celeste Classic} is designed for humans, unlike the environments from, for example, the \texttt{gymnasium} \cite{gymnasium} library.
	
	\noindent
	2: It runs on the PICO-8 \cite{pico8}, which allows us to modify its code. This grants us a reliable way to interface with the game interface. This is not true of \textit{Celeste} (2018) --- writing a wrapper for \textit{Celeste} would take a significant amout of time.

	\noindent
	3: The action space of \textit{Celeste Classic} is small, especially when ineffective actions are pruned.
\end{minipage}\hfill
\begin{minipage}{0.48\textwidth}
	\begin{center}
	\includegraphics[width=0.9\textwidth]{celeste}

	\vspace{1mm}
	\begin{minipage}{0.8\textwidth}
		The first stage of \textit{Celeste} (2018), showing the player dashing to the upper-right.
	\end{minipage}
	\end{center}
\end{minipage}

\vspace{5mm}

\noindent
When we started this project, our goal was to develop an agent that would learn to finish the first stage of this game. It starts in the bottom-left corner of the stage, and needs to reach the top right. If the agent touches the spikes at the bottom of the stage, the game is reset and the agent must try again.


To achieve this end, our agent selects one of nine actions (listed below) at every time step. It does this using a Q-learning algorithm, which is described in detail later in this paper.

\noindent
\begin{minipage}{0.5\textwidth}
Possible actions:
\begin{itemize}
	\item \texttt{left}: move left
	\item \texttt{right}: move right
	\item \texttt{jump-l}: jump left
	\item \texttt{jump-r}: jump right
	\item \texttt{dash-l}: dash left
	\item \texttt{dash-r}: dash right
	\item \texttt{dash-u}: dash up
	\item \texttt{dash-ru}: dash right-up
	\item \texttt{dash-lu}: dash left-up
\end{itemize}
\end{minipage}\hfill
\begin{minipage}{0.48\textwidth}
	\vspace{3ex}
	\begin{center}
		\includegraphics[width=0.48\textwidth]{jump}
		\includegraphics[width=0.48\textwidth]{dash}

		\vspace{1mm}
		\begin{minipage}{0.9\textwidth}
			The first stage of \textit{Celeste Classic}. Two possible actions our agent can take are shown: \texttt{jump-r} followed by \texttt{dash-lu}.
		\end{minipage}
	\end{center}
\end{minipage}


\vfill{}

This task has no direct practical applications. However, by developing an agent that completes this task, we will explore possible techniques and modifications to the traditional DQN algorithm, and we will learn how a simple machine learning model can be adjusted for a rather complicated task.

\vfill
\pagebreak