{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "source": [ "# Parameter-shift rules\n", "\n", "The output of a variational circuit (i.e., the expectation of an observable) can be written as a “quantum function” $f(\\theta)$ parametrized by $\\theta = \\theta_1, \\theta_2, \\dots$. The partial derivative of $f(\\theta)$ can in many cases be expressed as a linear combination of other quantum functions. Importantly, these other quantum functions typically use the same circuit, differing only in a shift of the argument. This means that partial derivatives of a variational circuit can be computed by using the same variational circuit architecture.\n", "\n", "Recipes of how to get partial derivatives by evaluated parameter-shifted instances of a variational circuit are called parameter-shift rules, and have been first introduced to quantum machine learning in Mitarai et al. (2018), and extended in Schuld et al. (2018).\n", "\n", "![](images/gradients2.png)\n", "\n" ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "source": [ "\n", "\n", "Making a rough analogy to classically computable functions, this is similar to how the derivative of the function $f(x)=\\sin(x)$ is identical to $\\frac{1}{2}\\sin(x+\\frac{\\pi}{2}) - \\frac{1}{2}\\sin(x-\\frac{\\pi}{2})$. So the same underlying algorithm can be reused to compute both $\\sin(x)$ and its derivative (by evaluating at $x\\pm\\frac{\\pi}{2}$). This intuition holds for many quantum functions of interest: the same circuit can be used to compute both the quantum function and the gradient of the quantum function (This should be contrasted with software which can perform automatic differentiation on classical simulations of quantum circuits, such as Strawberry Fields).\n", "\n", "\n", "## A more technical explanation\n", "\n", "Quantum circuits are specified by a sequence of gates. The unitary transformation carried out by the circuit can thus be broken down into a product of unitaries:\n", "\n", "$$\n", "U(x; \\theta) = U_N(\\theta_{N}) U_{N-1}(\\theta_{N-1}) \\cdots U_i(\\theta_i) \\cdots U_1(\\theta_1) U_0(x).\n", "$$\n", "\n", "Each of these gates is unitary, and therefore must have the form $U_{j}(\\gamma_j)=\\exp{(i\\gamma_j H_j)}$ where $H_j$ is a Hermitian operator which generates the gate and $\\gamma_j$ is the gate parameter. We have omitted which wire each unitary acts on, since it is not necessary for the following discussion.\n", "\n", "Note:\n", "\n", "In this example, we have used the input $x$ as the argument for gate $U_0$ and the parameters $\\theta$ for the remaining gates. This is not required. Inputs and parameters can be arbitrarily assigned to different gates." ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "source": [ "\n", "### A single parameterized gate\n", "\n", "Let us single out a single parameter $\\theta_i$ and its associated gate $U_i(\\theta_i)$. For simplicity, we remove all gates except $U_i(\\theta_i)$ and $U_0(x)$ for the moment. In this case, we have a simplified quantum circuit function\n", "\n", "$$\n", "f(x; \\theta_i) = \\langle 0 | U_0^\\dagger(x)U_i^\\dagger(\\theta_i)\\hat{B}U_i(\\theta_i)U_0(x) | 0 \\rangle = \\langle x | U_i^\\dagger(\\theta_i)\\hat{B}U_i(\\theta_i) | x \\rangle.\n", "$$\n", "\n", "For convenience, we rewrite the unitary conjugation as a linear transformation $\\mathcal{M}_{\\theta_i}$ acting on the operator $\\hat{B}$:\n", "\n", "$$\n", "U_i^\\dagger(\\theta_i)\\hat{B}U_i(\\theta_i) = \\mathcal{M}_{\\theta_i}(\\hat{B}).\n", "$$\n", "\n", "The transformation $\\mathcal{M}_{\\theta_i}$ depends smoothly on the parameter $\\theta_i$, so this quantum function will have a well-defined gradient:\n", "\n", "$$\n", "\\nabla_{\\theta_i}f(x; \\theta_i) = \\langle x | \\nabla_{\\theta_i}\\mathcal{M}_{\\theta_i}(\\hat{B}) | x \\rangle \\in \\mathbb{R}.\n", "$$\n", "\n", "The key insight is that we can, in many cases of interest, express this gradient as a linear combination of the same transformation $\\mathcal{M}$, but with different parameters. Namely,\n", "\n", "$$\n", "\\nabla_{\\theta_i}\\mathcal{M}_{\\theta_i}(\\hat{B}) = c[\\mathcal{M}_{\\theta_i + s}(\\hat{B}) - \\mathcal{M}_{\\theta_i - s}(\\hat{B})],\n", "$$\n", "\n", "where the multiplier $c$ and the shift $s$ are determined completely by the type of transformation $\\mathcal{M}$ and independent of the value of $\\theta_i$.\n", "\n", "Note\n", "\n", "While this construction bears some resemblance to the numerical finite-difference method for computing derivatives, here $s$ is finite rather than infinitesimal." ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "source": [ "### Multiple parameterized gates\n", "\n", "To complete the story, we now go back to the case where there are many gates in the circuit. We can absorb any gates applied before gate i into the initial state: |\\psi_{i-1}\\rangle = U_{i-1}(\\theta_{i-1}) \\cdots U_{1}(\\theta_{1})U_{0}(x)|0\\rangle. Similarly, any gates applied after gate i are combined with the observable $\\hat{B}: \\hat{B}_{i+1} = U_{N}^\\dagger(\\theta_{N}) \\cdots U_{i+1}^\\dagger(\\theta_{i+1}) \\hat{B} U_{i+1}(\\theta_{i+1}) \\cdots U_{N}(\\theta_{N})$.\n", "\n", "With this simplification, the quantum circuit function becomes\n", "\n", "$$\n", "f(x; \\theta) = \\langle \\psi_{i-1} | U_i^\\dagger(\\theta_i) \\hat{B}_{i+1} U_i(\\theta_i) | \\psi_{i-1} \\rangle = \\langle \\psi_{i-1} | \\mathcal{M}_{\\theta_i} (\\hat{B}_{i+1}) | \\psi_{i-1} \\rangle,\n", "$$\n", "\n", "and its gradient is\n", "\n", "$$\n", "\\nabla_{\\theta_i}f(x; \\theta) = \\langle \\psi_{i-1} | \\nabla_{\\theta_i}\\mathcal{M}_{\\theta_i} (\\hat{B}_{i+1}) | \\psi_{i-1} \\rangle.\n", "$$\n", "\n", "This gradient has the exact same form as the single-gate case, except we modify the state $|x\\rangle \\rightarrow |\\psi_{i-1}\\rangle$ and the measurement operator $\\hat{B}\\rightarrow\\hat{B}_{i+1}$. In terms of the circuit, this means we can leave all other gates as they are, and only modify gate $U(\\theta_i)$ when we want to differentiate with respect to the parameter $\\theta_i$.\n", "\n", "Note\n", "\n", "Sometimes we may want to use the same classical parameter with multiple gates in the circuit. Due to the product rule, the total gradient will then involve contributions from each gate that uses that parameter." ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "source": [ "\n", "### Pauli gate example\n", "\n", "Consider a quantum computer with parameterized gates of the form\n", "\n", "$$\n", "U_i(\\theta_i)=\\exp\\left(-i\\tfrac{\\theta_i}{2}\\hat{P}_i\\right),\n", "$$\n", "\n", "where $\\hat{P}_i=\\hat{P}_i^\\dagger$ is a Pauli operator.\n", "\n", "The gradient of this unitary is\n", "\n", "$$\n", "\\nabla_{\\theta_i}U_i(\\theta_i) = -\\tfrac{i}{2}\\hat{P}_i U_i(\\theta_i) = -\\tfrac{i}{2}U_i(\\theta_i)\\hat{P}_i .\n", "$$\n", "\n", "Substituting this into the quantum circuit function $f(x; \\theta)$, we get\n", "\n", "$$\n", "\\begin{align}\n", " \\nabla_{\\theta_i}f(x; \\theta) = &\n", " \\frac{i}{2}\\langle \\psi_{i-1} | U_i^\\dagger(\\theta_i) \\left( P_i \\hat{B}_{i+1} - \\hat{B}_{i+1} P_i \\right) U_i(\\theta_i)| \\psi_{i-1} \\rangle \\\\\n", " = & \\frac{i}{2}\\langle \\psi_{i-1} | U_i^\\dagger(\\theta_i) \\left[P_i, \\hat{B}_{i+1}\\right]U_i(\\theta_i) | \\psi_{i-1} \\rangle,\n", "\\end{align}\n", "$$\n", "\n", "where $[X,Y]=XY-YX$ is the commutator.\n", "\n", "We now make use of the following mathematical identity for commutators involving Pauli operators (Mitarai et al. (2018)):\n", "\n", "$$\n", "\\left[ \\hat{P}_i, \\hat{B} \\right] = -i\\left(U_i^\\dagger\\left(\\tfrac{\\pi}{2}\\right)\\hat{B}U_i\\left(\\tfrac{\\pi}{2}\\right) - U_i^\\dagger\\left(-\\tfrac{\\pi}{2}\\right)\\hat{B}U_i\\left(-\\tfrac{\\pi}{2}\\right) \\right).\n", "$$\n", "\n", "Substituting this into the previous equation, we obtain the gradient expression\n", "\n", "$$\n", "\\begin{align}\n", " \\nabla_{\\theta_i}f(x; \\theta) = & \\hphantom{-} \\tfrac{1}{2} \\langle \\psi_{i-1} | U_i^\\dagger\\left(\\theta_i + \\tfrac{\\pi}{2} \\right) \\hat{B}_{i+1} U_i\\left(\\theta_i + \\tfrac{\\pi}{2} \\right) | \\psi_{i-1} \\rangle \\\\\n", " & - \\tfrac{1}{2} \\langle \\psi_{i-1} | U_i^\\dagger\\left(\\theta_i - \\tfrac{\\pi}{2} \\right) \\hat{B}_{i+1} U_i\\left(\\theta_i - \\tfrac{\\pi}{2} \\right) | \\psi_{i-1} \\rangle.\n", "\\end{align}\n", "$$\n", "\n", "Finally, we can rewrite this in terms of quantum functions:\n", "\n", "$$\n", "\\nabla_{\\theta}f(x; \\theta) = \\tfrac{1}{2}\\left[ f(x; \\theta + \\tfrac{\\pi}{2}) - f(x; \\theta - \\tfrac{\\pi}{2}) \\right].\n", "$$" ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "source": [ "\n", "### Gaussian gate example\n", "\n", "For quantum devices with continuous-valued operators, such as photonic quantum computers, it is convenient to employ the Heisenberg picture, i.e., to track how the gates $U_i(\\theta_i)$ transform the final measurement operator $\\hat{B}$.\n", "\n", "As an example, we consider the Squeezing gate. In the Heisenberg picture, the Squeezing gate causes the quadrature operators \\hat{x} and \\hat{p} to become rescaled:\n", "\n", "$$\n", "\\begin{align}\n", " \\mathcal{M}^S_r(\\hat{x}) = & S^\\dagger(r)\\hat{x}S(r) \\\\\n", " = & e^{-r}\\hat{x}\n", "\\end{align}\n", "$$\n", "\n", "and\n", "\n", "$$\n", "\\begin{align}\n", " \\mathcal{M}^S_r(\\hat{p}) = & S^\\dagger(r)\\hat{p}S(r) \\\\\n", " = & e^{r}\\hat{p}.\n", "\\end{align}\n", "$$\n", "\n", "Expressing this in matrix notation, we have\n", "\n", "$$\n", "\\begin{align}\n", " \\begin{bmatrix}\n", " \\hat{x} \\\\\n", " \\hat{p}\n", " \\end{bmatrix}\n", " \\rightarrow\n", " \\begin{bmatrix}\n", " e^{-r} & 0 \\\\\n", " 0 & e^r\n", " \\end{bmatrix}\n", " \\begin{bmatrix}\n", " \\hat{x} \\\\\n", " \\hat{p}\n", " \\end{bmatrix}.\n", "\\end{align}\n", "$$\n", "\n", "The gradient of this transformation can easily be found:\n", "\n", "$$\n", "\\begin{align}\n", " \\nabla_r\n", " \\begin{bmatrix}\n", " e^{-r} & 0 \\\\\n", " 0 & e^r\n", " \\end{bmatrix}\n", " =\n", " \\begin{bmatrix}\n", " -e^{-r} & 0 \\\\\n", " 0 & e^r\n", " \\end{bmatrix}.\n", "\\end{align}\n", "$$\n", "\n", "We notice that this can be rewritten this as a linear combination of squeeze operations:\n", "\n", "$$\n", "\\begin{align}\n", " \\begin{bmatrix}\n", " -e^{-r} & 0 \\\\\n", " 0 & e^r\n", " \\end{bmatrix}\n", " =\n", " \\frac{1}{2\\sinh(s)}\n", " \\left(\n", " \\begin{bmatrix}\n", " e^{-(r+s)} & 0 \\\\\n", " 0 & e^{r+s}\n", " \\end{bmatrix}\n", " -\n", " \\begin{bmatrix}\n", " e^{-(r-s)} & 0 \\\\\n", " 0 & e^{r-s}\n", " \\end{bmatrix}\n", " \\right),\n", "\\end{align}\n", "$$\n", "\n", "where $s$ is an arbitrary nonzero shift (Note: In situations where no formula for automatic quantum gradients is known, one can fall back to approximate gradient estimation using numerical methods.\n", ").\n", "\n", "As before, assume that an input y has already been embedded into a quantum state $|y\\rangle = U_0(y)|0\\rangle$ before we apply the squeeze gate. If we measure the $\\hat{x}$ operator, we will have the following quantum circuit function:\n", "\n", "$$\n", "f(y;r) = \\langle y | \\mathcal{M}^S_r (\\hat{x}) | y \\rangle.\n", "$$\n", "\n", "Finally, its gradient can be expressed as\n", "\n", "$$\n", "\\begin{align}\n", " \\nabla_r f(y;r) = & \\frac{1}{2\\sinh(s)} \\left[\n", " \\langle y | \\mathcal{M}^S_{r+s} (\\hat{x}) | y \\rangle\n", " -\\langle y | \\mathcal{M}^S_{r-s} (\\hat{x}) | y \\rangle \\right] \\\\\n", " = & \\frac{1}{2\\sinh(s)}\\left[f(y; r+s) - f(y; r-s)\\right].\n", "\\end{align}\n", "$$\n", "\n", "Note\n", "\n", "For simplicity of the discussion, we have set the phase angle of the Squeezing gate to be zero. In the general case, Squeezing is a two-parameter gate, containing a squeezing magnitude and a squeezing angle. However, we can always decompose the two-parameter form into a Squeezing gate like the one above, followed by a Rotation gate.\n", "\n", "In physical experiments, it is beneficial to choose s so that the additional squeezing is small. However, there is a tradeoff, because we also want to make sure $\\frac{1}{2\\sinh(s)}$ does not blow up numerically.\n" ], "metadata": { "collapsed": false } }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Quantum gradients with backpropagation\n", "\n", "\n", "Using backpropagation can speed up training of quantum circuits compared to the parameter-shift rule---if you are using a simulator.\n", "\n", "\n", "In PennyLane, any quantum device, whether a hardware device or a\n", "simulator, can be trained using the parameter shift rule to compute quantum gradients. Indeed, the parameter-shift\n", "rule is ideally suited to hardware devices, as it does not require any knowledge about the internal workings of the device; it is sufficient to treat the device as a \\'black box\\', and to query it with different\n", "input values in order to determine the gradient.\n", "\n", "When working with simulators, however, we *do* have access to the internal (classical) computations being performed. This allows us to take advantage of other methods of computing the gradient, such as\n", "backpropagation, which may be advantageous in certain regimes. In this tutorial, we will compare and contrast the parameter-shift rule against backpropagation, using the PennyLane\n", "`default.qubit ` device." ], "outputs": [] }, { "cell_type": "markdown", "source": [ "\n", "## The parameter-shift rule example\n", "\n", "The parameter-shift rule states that, given a variational quantum circuit $U(\\boldsymbol \\theta)$ composed of parametrized Pauli rotations, and some measured\n", "observable $\\hat{B}$, the derivative of the expectation value\n", "\n", "$$\\langle \\hat{B} \\rangle (\\boldsymbol\\theta) =\n", "\\langle 0 \\vert U(\\boldsymbol\\theta)^\\dagger \\hat{B} U(\\boldsymbol\\theta) \\vert 0\\rangle$$\n", "\n", "with respect to the input circuit parameters $\\boldsymbol{\\theta}$ is\n", "given by\n", "\n", "$$\\nabla_{\\theta_i}\\langle \\hat{B} \\rangle(\\boldsymbol\\theta)\n", " = \\frac{1}{2}\n", " \\left[\n", " \\langle \\hat{B} \\rangle\\left(\\boldsymbol\\theta + \\frac{\\pi}{2}\\hat{\\mathbf{e}}_i\\right)\n", " - \\langle \\hat{B} \\rangle\\left(\\boldsymbol\\theta - \\frac{\\pi}{2}\\hat{\\mathbf{e}}_i\\right)\n", " \\right].$$\n", "\n", "Thus, the gradient of the expectation value can be calculated by\n", "evaluating the same variational quantum circuit, but with shifted\n", "parameter values (hence the name, parameter-shift rule!).\n", "\n", "Let\\'s have a go implementing the parameter-shift rule manually in\n", "PennyLane.\n" ], "metadata": { "collapsed": false } }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import pennylane as qml\n", "from pennylane import numpy as np\n", "from matplotlib import pyplot as plt\n", "\n", "# set the random seed\n", "np.random.seed(42)\n", "\n", "# create a device to execute the circuit on\n", "dev = qml.device(\"default.qubit\", wires=3)\n", "\n", "@qml.qnode(dev, diff_method=\"parameter-shift\")\n", "def circuit(params):\n", " qml.RX(params[0], wires=0)\n", " qml.RY(params[1], wires=1)\n", " qml.RZ(params[2], wires=2)\n", "\n", " qml.broadcast(qml.CNOT, wires=[0, 1, 2], pattern=\"ring\")\n", "\n", " qml.RX(params[3], wires=0)\n", " qml.RY(params[4], wires=1)\n", " qml.RZ(params[5], wires=2)\n", "\n", " qml.broadcast(qml.CNOT, wires=[0, 1, 2], pattern=\"ring\")\n", " return qml.expval(qml.PauliY(0) @ qml.PauliZ(2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let\\'s test the variational circuit evaluation with some parameter\n", "input:\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parameters: [0.37454012 0.95071431 0.73199394 0.59865848 0.15601864 0.15599452]\n", "Expectation value: -0.11971365706871566\n" ] } ], "source": [ "# initial parameters\n", "params = np.random.random([6], requires_grad=True)\n", "\n", "print(\"Parameters:\", params)\n", "print(\"Expectation value:\", circuit(params))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also draw the executed quantum circuit:\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": "
", "image/png": "iVBORw0KGgoAAAANSUhEUgAABGAAAAGjCAYAAABjSmMUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy89olMNAAAACXBIWXMAAA9hAAAPYQGoP6dpAACETUlEQVR4nO3deWBU9bn/8c8smZnsCyELJLKGzbCo4AauiKBoRS2gqL323vZX7W5dullrrbVa0d62t9b22qtWjYLV4lJFcQEFN0CWRLYAAgkhCYQsZJnJLOf3R0gkkpBJMpOTmbxff4UzZ3lmIPM5POd7vsdiGIYhAAAAAAAAhI3V7AIAAAAAAACiHQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNnNLgDoa/X19aqsrJTb7Zbb7Ta7nHZcLpdcLpcyMjKUkJBgdjkAgGOQHwCAniA/0IoGDKLa7t27tWzZMhUVFWn7jmIVF+/Uwcpys8sKyuCMLOXljdbYMXnKz8/XvHnzNHLkSLPLAoABgfwAAPQE+YETsRiGYZhdBBBKpaWleuaZZ/TskqXatOFT2WKccmYMl5KyFJM6RPa0IbLFp8pid8pis0sWi9kltzAMGX6fDJ9H/oZq+Q6XyVtdJtWVy1O5R36vR5NPOVXXLlyg6667Tjk5OWZXDABRhfwAAPQE+YFg0YBB1HC73XrwwQd172/uk98w5Bo5VbFjZih21DRZHS6zy+uVQHOTmnatU9OO1XLvXiebxaJf3Plz3XbbbXK5Ivu9AYDZyA8AQE+QH+guGjCICitWrNA3v3WT9u3bq8SpVyr5rAWyOuPMLissAp5G1X64VPXr/qXck4bpf//6qGbNmmV2WQAQkcgP8gMAeoL8ID96ggYMIt6SJUu0aNEiuU6aqJSZNykmPdfskvqE91CJat5+VO59hSooKNDChQvNLgkAIgr5QX4AQE+QH+RHT9GAQURr/fKLm3C+0i75gSxWm9kl9Skj4Nfh1/+gxi0r+RIEgG4gP8gPAOgJ8oP86A0aMIhYK1as0Jw5cwbsl1+rY78Ely9fznBAAB0KBAKqdXslScmuGFmtVpMrMg/50YL8ABAM8uML0ZQfvvrDstgdsrm6/+hp8qPnaMAgIrndbo2bcLIOGolKn39PRH/5hYIR8OvQ83dpsOWItm35jImxgAGsttGrorJabS6tVdH+WhXur1VFnVseX6Ddek67VZlJLk0cmqyJOcmaODRZ+UOSlRwXY1LlfYP8aI/8ANCK/DixaMqP5srdqnz+V5Kkod9+QpYePJWJ/OgZu9kFAD2xePFilezbp6yv/ymiv/xCxWK1KWXmTSp54rtavHix7rzzTrNLAtCHdh2s19Mf7dXbWyu173BjUNt4fAHtO9yofYcb9e/CA23LT0qL08zxGbr+zGEaNbj7V8X6O/KjPfIDGNjIj+BFS3407Vqrgy//TkZzkxzZY3vUfJHIj55iBAwiTmlpqUaNzpNrymVKPf9Gs8vpV6pXPi7Ppte0s3iHcnJyzC4HQBj5/AG9tbVCT320V2t2VrV7LTcttuXK5NAUTRyarGGD4mQY0rkPvitJeu/2C2SxSHurGlW4v1aF+2tUuL9WJYeb2u1n+uhBuuHMYbpofKbstsgfck5+dI78AAYO8qP7oiU/6ta/ouq3/1exo6bJc2C74sefp7SZ3+zVPsmP7mEEDCLOM888I79hKPmsBWaX0u8kn7VQZRteVUFBge644w6zywEQBm6vX39f/bme+nCvyuvckiSrRbpwXKaumZarqcNTlRLnOG67xmZf28/piQ7FOezKTYvTjLz0tuU1jc1at6daz60t0TvbKrRmZ5XW7KxSVpJLN5w1TN84Z4Sc9si96kd+dI78AKIf+dFzkZ4fRsCv6nf+riPrX1bi1CuUdPpV2v/If8g5ZFyv901+dA8jYBBxppx6moobYzV43k/NLqVfOrTst8qLd2vD+nVmlwIgxDaW1Oj25zepuLJekjQo3qFrTs/VtaefpJzUuBNu29js04S73pAkbblntuIcJ74GU1rdqGc/2afnPilRVUOzJCkvI0GL50/W5NyU3r8ZE5AfJ0Z+ANGL/OidSM6PQHOTDr3yoJp2rVPaRd9S4qlz1bjjQx3812809Ob/kz0po9fHID+CF/njwTCg7N69W5s2fKq4sTPMLqXfih07XRs/Xa/du3ebXQqAEHF7/br/9W266pE1Kq6sV3qCUw8vmKwPfnqhbp89rsuT557ISY3T7bPH6YOfXqiHF0xWeoJTxZX1uvKRNXpg+Ta5vf6QHzOcyI+ukR9A9CE/ei+S88N3pEoVBT+Re1+hMq6+S4mnzpUkecq2yZaQJlvi4JAch/wIHg0YRJRly5bJFuNU7KhpZpfSb8WOmiZbjFMvvfSS2aUACIGNJTW67E+r9eiqXQoY0hVThmjFLefqqlNz+mQ4t9Nu01Wn5mjFLefqiilDFDCkv6zcpcv/tFobS2rCfvxQIT+6Rn4A0YX8CI1IzY/myt0qf+pW+RtrlXXdA4odNbXtNc/+bXIOGdfjCXi/jPwIHg0YRJSioiI5M4bL6uAxZ52xOmLlzBimoqIis0sB0EuvFR7Q/Ec/0M6jVy3/esNp+sM1pyg1/vh79MMtNd6hP1xziv56w2lKT3CouLJe8x/9QK8f8wSM/oz86Br5AUQP8iN0IjE/mnatVfkzP5YtLllZNzwkR8bIttcMv0/N5TvlHNr7+V9akR/BowGDiLJ9R7GUlGV2Gf1fUlbLZwUgYi1dW6LvFnwqr9/QrAmZWnHLuZp9svnff7NPztKKW87TrAmZ8voNfafgUy1dV2J2WV0iP4JEfgARj/wIrUjLjyOfvqrKF34t10mTlLnoAdkTB7V7vfngHhk+jxxDxof2wORHUGjAIKIUF+9UTOoQs8vo92JSh6i4eKfZZQDooaXrSnTHC5sVMKSFU3P16PWnmXLVsjOp8Q49ev1pWjg1VwFDuuOfm/v9STT5ERzyA4hs5EfoRVJ+NH3+qQ6veFRxY6dr8JU/63DUjmf/VslqlzNrVEiPTX4EhwYMIkZ9fb0OVpbLnhYZX4BmsqcOUWXFATU0NJhdCoBuer3wgH7ywmZJ0tenD9f9V0+UzRqae7RDyWa16P6rJ+rr04dLkn7ywmYtL+qfw8nJj+CRH0DkIj9CL9Lyw56SLXtKthp3fKCa9/6hQHPTcet4yrbJkTlKFntoG3PkR3BowCBiVFZWSpJs8akmV9L/2RLSJH3xmQGIDBtLavT95zYoYEjXTMvVXZdNCNkEeeFgsVh012UT2q5kfu/ZDdrUDydWJD+CR34AkYn8CI9Iy4+Y1GwN+a8/K3n6tTqy/lWV/vlrqv34BRmG0bZO8/5tIZ3/pRX5EZwTP8Qd6EfcbrckyWJ3hv1YvtoK7X/0v9ovtNpki0uWc+gEJZ1xtZzZeW0vBTyNKvu/78rfUK3sG/8gR/pJx+2z9qN/qmbVE0qYdLEGXfL9sNbf2tFu/cwA9H9ur1+3Pb+p7Z7931w5sV+fPLeyWCy676qJOtzYrBVbKnTb85v0yvdmyBUT/idsBIv8CB75AUQe8iN8+jI/QsVidyjl7GsUP2a6yv5+s2pWPi73558q7aKbZI1NkK+2Qs4hoW/AkB/BYQQMIkbbF6Ct7/qG9pRsJU+/VsnTr1XS1CtkT8tR4/bVKn/6drlLvpjl2+qM06BLfyD5far69+9lBPzt9tN8cI9qVj8tW1KGUi/8RtjrtthiJPEFCESSP7xd3Pa0it9dPalfDhvvjM1q0QNXT2p7usUf3+5fk/CRH8EjP4DIQ36Ejxn5ESoVz/9SkhSff5F8tZUqe/y7OvTv30uSnEPHhvx45EdwIu9fEtCHHX17arZSZlzXblntR8+rZtWTqnn/aWUtur9teeywyUo8da6OfPqqaj9cqpTp10pqedTboVcflvx+pV/6Q1mdceEv/OhndOxwQwD918aSGv111S5J0m+uzO9XEyYGKy3eoXvnTdRNT6/Xo6t2afbJWZqcm2J2We2RH10jP4CIQn70ERNHFBkBv7yH98tb+bmaD+5RwF0vw+dtKcseI6srQY7BwxWTMUIxaUNlsdrkKd8pf13LrUDpc38ow9es2k9eVN2Hz8ueki1b4uDQF0p+BIUGDNBNCZMuVs2qJ9Vcfvws3ynn36imz9er9oMliht9hhyZI1W75ll5K3cr8bTL5Ro2yYSKAfRnbq9ftz+/SQFDumLKkH7xqNCempOfpa9MHqKXN5Xptuc36dXvz5DT3n+GkpuN/AAQSuRH9PIdqVLDZ++qcccaeQ/uleFrliTZEgbJFp/SNtrE8Hvlb6hRXX2VpJbbgGIGD1PzgZaRREP+39/alqecfY0SJs6S1ZUQEbeoRSsaMEBPWY8PBWuMS4MuvUUVBT/RoX8/rLSLb1btR8/LnpajlPNu7PsaAfR7f1/9uYqPDh2/+/KTzS6n1371lZP1wa4qFVfW67H3P9d3Lhhtdkn9D/kBIATIj+hiBPxqLP5IDZtXqOnzT2Wx2RU7+gzFTzhfjowRihk8XLbYpA639TfVyVu5R80HP1f9pjfblle/9TfFT5qluLwzZbHaZE8c1FdvB51gDhigm+o3vSFJcuVM6PB1V84EJU2bJ+/BPap47k5JUvrcW2SNiZzJuwD0DZ8/oKc+3CtJ+tml4yJy6PiXpcY79NNLWib3e/qjvfL5AyZX1H+QHwBChfyILp6y7Sr/x490aNlv5W86orSLb1bOd/6hwVf8WElTr5DrpEmdNl8kyRabJNewSUqYPFveQy3/LlIv+pb8TXU6tOy3Kv/Hj+Q5sKOv3g5OgBEwwAn4qg+oZvUzkiTD65GnfKc8+zbLGp+i1PP/s9Ptks++Rkc+/bcMn0eJp14m55DQT3QFIPK9tbVC5XVupSc4NHdSttnlhMxlk7N132tbdaDWrbe2VmpOfuQOi+8p8gNAOJEf0cHvrlfNqidVv3G5YjJGKOv6B+UcOr7H+yt77NuSpLQ531Pi5NlKOu1yefZvVdWbf1H5P25VwimXKOXcr8nmSgjVW0A30YABTsBXc0C1a55tt8wWn6rM6x5QTOqQTrer++RfMnweSVLT5+sV8LpljXGFtVYAkeepj1quUi2clhtV97o77TYtnJarR1bu0tMf7Y36E+iOkB8Awon8iHyeAzt08MV7FWhuUurMbyrx1LmydHCLatD7O2bi3cTJs9uWO4eOV/Z//F5HPv23at5/Sk3FH2nwVXfKmT2m1+8B3UcDBjgB14hTlbngHkmSv7FW9YVvq2bVEzr4wq+V9bWHZXXEHreNp3ynaj9aKntajuJGn666T15UzaonlXbRt/q6/AHjQG2T3i8+pKL9taqoa3n0XWaSS/lDk3VOXrqyk4//ewLMtrOyXmt2Vslqka49/SSzywm5RWecpL+s2qXVOw9p18F6jRo8sK62kR+RgfxAJCI/Il9j8cc69PLvFJMxXFnzfhaSuVnKn/yhpC8m3j2WxWpT0tSvKG7sdB1cdp8qCn6q9K/cobi8M3p9XHQPDRggSLa4ZCWfcZUMT4NqP1yimveeUtpF/6/dOobPq6pXH5YMQ+lzb5Ejc5SaPv9UR9a/qrix0+XKzTep+ui0rbxOD725Q29vrVDAkOxWi3yBlkfftf5stUgzx2fq1ovHaFxW5/fOAn3tmY9brl5eOC5TOal98HjhPpaTGqeZ4zL01tZKPf3RXv0yCiaI7Cnyo/8hPxDJyI/I1rBttQ69/DvF5Z2pQZfdGpJ5vmqOjrh0Dp1wwlGW9sRByrzmPlW9+pAOLrtP6ZffrvhxM3p9fASPSXiBbko6a4FsCWk6suE1+Wor2r1W8/5T8lbtU9IZV8s5ZKwsNrvS5/5IstpU9dofFPC6Tao6ugQChv787k7N/eNqvbOtUkfPmdtOno/9OWBI72yr1Nw/rtaf392pwDHrAGZ6e2vLMOFrpuWaXEn4LJzWcmX2nW2VJlfSP5Af5iM/EA3Ij8jlLilqab6MP0fpV/w4JM2XgNet2qNzjmVee1+X61tjnEq/4seKGzdDh17+ndwlRb2uAcEbEA2YtWvX6tJLL1VKSori4+N15plnaunSpWaXhQhljXEq6YyvSgGfatY817bcXbpFdWuXKWbwcKXMWNS23JE5UslnLZCv5oBqVj5hQsXRJRAw9JMXC/XgG9vlDxjyB3FC3Lreg29s109eLOQkGqarbfRq3+FGSdLU4akmVxM+U4e1vLe9VY2qbfKaXI35yA9zkR+IBuRH5PI3HdGhVxbLmTNB6XN/1Kv5Xo517MS7FltwN7hYrDalz/2RnEPH69ArD8nfdCQktaBrUd+AeffddzV9+nStXr1aCxYs0E033aTy8nItXLhQDz30kNnlIUIlTpkjW0KaGj57R97qAwo0u1X12u8li1Xpc2+RxRbTbv3ksxfKkTlKRz79t9z7Ck2qOjr8ZdUuLV1X0uPtl64r0V9W7QphRUD3FZXVSpJy02KVEhf5jw7tTGq8QzmpLXNofLa/1uRq+gfywzzkB6IB+RGZDMNQ1fI/yvC6lX7ZrSFrvnQ28W4wLFab0i+/VYa3SYeX/0mGQYO5L0R1A8bn8+mb3/ymrFar3nvvPf3tb3/TQw89pE2bNmnMmDH62c9+pr1795pdJiKQxe5Q0pnzpYBftWsKVL3ycfmqD7SdKB+3vtWmQXNvkWw2Vb323wo0M5S8J7aV1+nhFTt6vZ+HV+zQtvK6EFQE9Mzm0paTyYlDk02uJPwm5bS8x81RcAIdCuSHOcgPRAvyIzLVb16hph0fatAl35c9aXDI9nuiiXeDYU/K0KA531fjjg9Uv3lFyOpC56J6Et533nlHu3bt0te//nVNmTKlbXlycrJ+9rOf6cYbb9STTz6pu+66y7wi0S/ZkzM17MevnnCdpNMuV9Jpl7f9edDFN59wfcfg4Rp227JQlDdgPfRm70+eWz385g797WtTQ7Y/oDuK9reeQKeYW0gfyB+arNcKy1UYBSfQwSA/+ifyA9GC/Ig8RsCv2g+eU9z4cxU35uyQ7TfYiXe7Ejf2bMWNO0e1HzynhIkzQzY6Bx2L6hEwK1eulCRdfPHFx702e3bLEK1Vq1b1ZUkAeuhAbZPe2loR1D37XfEHDL21tUIHaptCUBnQfYX7B84VzNb3WBThJ9CIXOQHogn5EXkat6+Rv65SyWd8NWT77O7Eu11JPvOr8tdVqnH7B73eF04sqhswxcXFkqS8vLzjXsvKylJCQkLbOgD6t/eLDymUt6YGDGl18aHQ7RDohoq6lttIhg2KvseHftnwQfGSvnjPQF8jPxBNyI/IYhiG6j75l1zDpsiROTJk++3JxLsn4sgcJdewyapb+yJzwYRZVN+CVFvb0i1NTu64Q5yUlNS2TqQwDENNXr/ZZZiiyeszu4SI0+T1qbE5Oj63jfuqZbda2j0qtDfsVos2lNRo7qTskOwPCFYgEJDHF5AkGYb67Hf02OP05fdC4OiJnNsbUL27WVZr31/7IT+6j/zoHPkBs5AfkZcfzQd2qLm8WBnzfxWiino38e6JJE27UpX/vFvN5cVyZo8J2X7RXlQ3YKJRk9evCXe9YXYZpvCU7zS7hIjz1b98KGfWQbPL6Jd8AUMFH+9Twcf7zC4FA9i5D75rynGn3vu2KcfNv9ucCf7Ij+4jPzpHfqA/ID/6Rm/zw13ymSwxTrmGTwlNQer9xLudcY04RRa7U56SIhowYRTVtyC1jnzpbJRLXV1dp6NjAAAAAADoqeaybXJk5YVsYttQTbzbEYvVJkd2njxl20O6X7QX1SNgWud+KS4u1mmnndbutfLyctXX1+v00083o7Qei42xacs9oRtqFkk2bBisGU+aXUVk+efNZ+mUU041u4yQuPfVLVq6rjSkQ8gXTMvVnXPHh2R/QLACgUDblbz3br9A6YmOPjluY7Ov7crlujtnKs7RN6cAB494dN6DKyVJRXfPMmUIOfnRfeRH58gPmIX8iLz88JRtV/zJF4SkllBPvNsR55CxatjCQ2rCKaobMOedd55++9vf6s0339Q111zT7rU33nijbZ1IYrFY+uxLr7+JjRmY77s3YmPsUfPvZcpJqSr4pCRk+/MFDJ2SmxI1nw8ii9NulccXkMUiU/4Nxjn67rvBammWJLlirEpw9c1/Fr6M/Og+8qNz5AfMRH70rd7kh7+hWv76Kjmyj38gTE+U/f07kkI38W5HHNljVPfxC/I3VMsWnxqWYwx0UX0L0syZMzVy5EgVFBRo48aNbctra2t13333yeFw6Gtf+5p5BSLsDMPQgSd+oIolvzC7lB45sukN7f3dV9R8cI/ZpZjunLx0WS2h25/VIp2TNzh0OwS6ITPJJUnaW9VociXht6eqQdIX7zlSRHp+VL/3lPb9fr78DdVml2I68gPRhPyIHAF3vSTJFtf7KS885Tvlr62QFNqJd7+stdaAuyFsxxjoorp1b7fb9dhjj2n27Nk699xzdc011ygxMVEvvPCC9u7dq8WLF2v48OFml4kwaih6R80Vu5R1/eJ2yw2fV7Uf/1MNn70jX90h2WITFTtqmlLOuUG2+JSg9m34mlW96kk1l++Ur+aA/E1HZHXFKyYlWwmTLlb8yRcc153e+8BlXe536M2Py57UcmKXkD9TdR8uVfW7/6fMBfcE96ajVHZyrGaOz9Q72yrl7+UwcpvVopnjMpSVHJmBjsg3cWiy9h1uVOH+Ws3ISze7nLAq3N8yD1v+0Miacy2c+XGsxh0f6Minr6m5YpcCXrdsCWlyDhmr1PO/3pYFrQKeRtWsfkaNOz44enUyTfHjZih5+rWyOmLbrZt0+pU6sv4V1ax+RoNmf7fbdUUT8gPRhPyIHBa7Q7F5Z4ZkJEm4Jt79Mlt8qmLzzpTFbs6Io4EgqhswknTBBRdo9erV+uUvf6klS5bI6/Vq4sSJeuCBB7Rw4UKzy0MYGUZANWsK5Mw5Wc6h49otr3zx13J//qkcQ8Yqacx0+arLVL/pTbn3blLWDQ8F1akONDepfuPrcmSPUezIqbLGJSvgrlfT7vWqev0Patj6njIW/EoWyxcDzZKnX9vhvnzVB9SwZaViBp3U7oTbYrMrcdo8Vb/1V7lLt8iVM6EXn0jku/XiMXpnW2VI9vWji5ndHebJH5qsfxceUOH+GrNLCbuioyfQEyPoBDrc+dGyL0OH3/iz6jctlz0lW3Hjz5XVESt/fZXcJUXy1VW2y4NAs1vlBT+Rt3K3XMNPUfz489RcsUt1n7wod0mhshY90O6E2eZKUMLki3Vk3ctKPnOB7MkZofuAIhD5gWhBfkQOe3KmMq66s9f7CefEu18WkzY0JDWjc1HfgJGk008/Xa+//rrZZaCPNe1eL39thZLPWtBueUPh23J//qnixp+n9Mtvk8XSMi75yIbXdPjNR1Tz3lMaNKfrq4XW2ETl/nCJLLaYdsuNgF8VS+6Ue88GNe1er7hR09peS5lxXYf7OrziUUlSwqRZx70WP/5cVb/zmOo3vj7gGzDjspL0o1lj9OAbvZud/UezxmhcVlKIqgK6b1JOy8lk69W9aLa5tOU9ToqgE+hw54ckHVn/suo3LVfCKXOVdtH/O+4JGUbA3+7PdR+/IG/lbiWd8VWlnn9j2/LqlU+o7uN/qm7tsuPqTTj5Ah1Zu0z1m95Qyrk3BPv2oxL5gWhBfgwsfTHxLvpWVM8Bg4GtYfMKSRbFjZ3ebnn9ppYJmFPP+4+2k2dJSphyiewpWWrYslIBr6fL/Vss1uOaL1LLI9zi8s6SJPmqy7rcj+FrVsOWlZLNrvj8C4973RaXLFfuRDVuX6NAc1OX+4t2N583Sgum5vZ4+wVTc/Xt80eFsCKg+/KHtJxMlhxuUk1js8nVhE91Q7NKq1u+t06OoBPocOdHwOtR7ZpnZU/J6rD5IqndMsMwVL/5DVkcsUo+u/1DBZLPvkYWR6zqN7953D4cmaNkT81WfdHbXdY0EJAfiAbkx8DSFxPvom/RgEFUMgxD7n2FsqcNlc2V8MVyX7M8B3bInpZz3HBsi8Ui1/ApMrxuNZcX9+LYATV9vl6SFJM+rMv1G3d8oIC7XnGjz+h06Lpz6LiW2vdv7XFd0cJqtej+qybq9tljZbNaZAtiZsXW9W6fPVb3XzWx3X+cADMkx8XopLQ4SdK6PdE7Seq6vS3vbdigOCXHHt+w7o/6Ij/cezYo4K5XbN6ZUiCgxu0fqPaj53Vkw2vydtC491WXyV9/WM6h42V1tJ97xOpwyTl0vHw15fLVHTxuW+eQcfIfOSTv4f3BfgRRi/xANCA/Bo6+mngXfYsGDKKSt6pEAfcRObNGt19efUAyAopJze5wu9b7KoMZudLK8HtVs/oZ1bz/jA6v+IvKHvu23LvXK37iRYodPqXL7VuvWiZMurjTdRxZLY+v85TSgJFaTqK/c8Fo/fv7MzRzXEbb0y3sx5xMt/5stUgzx2Xo39+foe9cMFrWUD4KA+iFmeNb/hP/3NrQPR63v1mydp8k6cJxkTP/SF/kR3P5TkktIynLHv+uDi67TzWrntThNx9R2f/epOp3/v6lY5e1O0Znx+6oefNFfmzpsq6BgPxANCA/Boa+mngXfYtxTIhK/iOHJEnWLz2RIuBpeWSf1Rnf4XYWR1y79YJh+H2qPTo51tG9KOn0q5Ry3n90ua23plzuvYWyJQ2Wa8Qpna7X+mQN39H3hRbjspL0t69N1YHaJq0uPqQNJTUq+LglsBdMy9UpuSmakZeu7OTYLvYE9L3rzxymx9fs0TvbKlRa3aic1DizSwqpksONevvopKfXn9n1aMD+oi/yw99YI0mqW7tMjqxRyvraw4oZlKvmil2qWv4/qlv7L9lTs5V4yqWSJMPT8jhQS2fHdsa1W+9Y5EfHyA9EMvIj+tV88Jykvpl4F32LETCISoGmI5I6P1EOJasjVsN+/KpOuuNlDb35CaVdfLPqN72hioKfdnki3jLPgKGEiRe1e1rSccc4Ogw+0FQXytKjRnZyrOZPzdWdc8e3Lbtz7njNn5rLyTP6rVGDEzR99CAFDOnZT/aZXU7IPfvJPhmGNGN0ukYNTuh6g36iT/LDaHkUssUWo8FX3iln9hhZHbFy5eZr8LyfSBar6j75V0gOZXUlSiI/OkN+IBKRH9Et4HWr9v2nJTHxbjSiAYOo1PooTsPvbbfc6my9Qnn8VUJJMpob263XrWNarLInpSvxlEuVNud78uzfotoPl3S6vmEEWiZGtFg7fPpRu3V9LZOsWWKc3a4LQP91w9Ere0vWlsjj83exduTw+PxacnRofKRdveyL/Ght7jiyRsueOKjda47Bw2VPyZSv5oAC7vqWmo6u39EIl5blje3Wa/ca+QFEJfIjejHxbnSjAYOoZD06mW3rlcxW9pQsyWJtuZe/A633z9t7OdQv9ujtRO59hZ2u4969Xv4jh+QaPkX2pBPf3xpwt7yPzibpBRCZLhqfqawklw7VN+vfmzv+XopEr246oKqGZmUnu3TR+Mi6f78v8sOeNrTlWK6OR9lYnUdHPR5tnpxojpdjl3c0TL0tP2LJDyCakB/RiYl3ox8NGEQlR/pJLSfKX3rqgzXGKUd2nnyHS+WrrWz3mmEYcu/ZKEuMq23Swp7yHzksSbJYO+9a129eIenEk++2an0fwTxVCUDksNusuuGslt/r+17bpuqGyH+kaHVDs377+jZJLVcv7bbIOtXoi/xwnTRJUsuEv19m+H3yVZfJEuNqa7rbU4fIlpAmz/6tCjS7260faHbLs3+r7MmZsicNPm5/3sOlkqSYwcO7rAtA5CA/ohMT70a/gfevGgOC1ZWgmMHD1VxeLMMItHstcfIcSVL1qidlHL0PX5LqN74uX0254iecL+sxQ7UNv0/eqpLjrno2H9qngLf9ibDUct/m4XcekyTFjpraYX3+xlo17vxE1rhkxeWd0eX78ZRtlyS5TprY5boAIss3zhmhvIwEHar36O5XPjO7nF775cuf6VC9R3kZCfrGOSPMLqfb+iI/YlKz5Rp+inzVB3Rk0xvtXqv96HkFPA2KG3OWLFabpJbHXCdMmi2juUm1RydmbFv/g+dkNDcpoZMrpZ6yHZLVJufQ8R2+DiBykR/RhYl3BwZuKkPUihtzlmpXPyPP/u1y5Xxx4hk/caYatr2vxq2rVF5bLlfuRPmqy9S440PZkzOVcu4N7fbjr69S2WM3y5aUoZyb/69teeO291W3dpmcORNkT86U1REnf32VmnavV6CpTs6ck5U49YoOa2soelsK+BR/8gWy2GJO+D4Mw5B77ybFDMpVzNFh6wCih9Nu04PzJ+uqR9bopY1lunRitmafnGV2WT2yvKhcL28qk81q0eL5k+W028wuqUfCnR+SlHbxt1X+9O06vPxPair+SDFpOWqu3CX33s2yJWUo9fz/bLd+0hlXq3HnR6r7+J9qrtglR9YoNZfvknvPBjmy8zrMm0Bzk5rLtss1fIqsDlcIPyEA/QH5ET2YeHfgYAQMIs8xVx1PJGHSxZLVpoYt77ZbbrFYlXHVL5Q8fZECjXWqW7dMnv1blTBplrJuWBz0PCuxo05X/Phz5a87qMYtq1S39l9q2r1eMYOHK232d5V57X3troQeq/X2o8RJXd/b6Skpkr/uoBKmzAmqLklfPGHDYgl+GwCmmZKbom+dN0qS9PN/FUXkUPLDDc26c1nLvFffOnekJuemmFtQR/pJfkgto2Cy/+P3is+/SM3lO1W3/hV5Dx9Q4qlzlf21h2VLSG23vtXhUtai+5U49Qp5q0pV98kyeQ+XKmnalcpc+JsO86Zx+wcyfB4lTrkk6LrIDyCykB99JMj86KmomHiX/AhKhP7tYiByuVqu3hl+X1Dr2xMHKX78uWrc+p5Sz7ux3ZMpLPYYpcxYpJQZi7reT3Kmhv341eOWO7Pz5Mzu2VwxQ77xl6DXrd/0hqyuRCXkzwx6m9and7R+ZgD6vx/MzNNbWypUXFmvO17YrEevP002a2ScxPgDhn78wmYdqm9WXkaCfnBR7+bRCrX+lh9trycNVvrcHwZVk9Ty9KS0md9U2sxvBrV+/eY3ZE8dothR04I+BvkBRB7yI3y6mx89YRhGVEy8S34EhxEwiBhtX4A+T9DbpJxzgwxfs4582vkJcH/mPbxfDVvfU/LZC2V1JQS9XetjR/kCBCKHK6ZlKHmMzaIVWyr0sxcL280z0l8ZhqGfvVioFVsq5LBZ++XQ8YGYH017NspTukWp59/YNpdMMMgPIPKQH+HTk/zoLovFosxF9yv3h0vDdoy+QH4EhwYMIkZGRsuj6PwN1UFvY0/O0KBLb5HFERuussLKd+SQkqdfq8RT53ZrO399y1OYWj8zAJFhSm6K/njNKbJapCXrSnTPq1v69Um0YRi659UtWrKuRFaL9Mdrp/TLoeMDMT8MT6NSL/hPxY05u1vbkR9AZCI/wqMn+dETrtz8dqMtIxH5ERxuQULESEhI0OCMLDUfLuvWdvHjzwlTReEXO2yyYodN7vZ2vuoyZWRmKz4+PgxVAQinSyZm6/6rJ+mOf27W42v2qNHj131XTex3w8n9AUM/fXGzlq5reczxA1dP0pz8bJOr6thAzI+4sd1rvLQiP4DIRX6EXk/zYyAiP4LDCBhElLy80fJW8wXYFW91mfLyRptdBoAeWjA1V7/76qS2K5k3Pb2+X02seLihWTc9vV5L15XKapEe/OokzZ+aa3ZZJ0R+BIf8ACIb+RF65EdwyI/g0IBBRBk7Jk+qKze7jP6vrrzlswIQsRZMzdWfF53adk//rN+v0vIi87//lheV6+Lfr9KKLRWKsVn0yHWn9vuTZ4n8CBr5AUQ88iO0yI8gkR9BoQGDiJKfny9P5R4FmpvMLqXfCjQ3yVO5V/n5+WaXAqCXLpmYredvOlujMxJ0qL7lquEPnttgytXM6oZmff/ZDbrp6fVtT6v4501n99th419GfnSN/ACiB/kROuRH18iP4NGAQUSZN2+e/F6PmnatM7uUfqtp11r5vR7NmzfP7FIAhMCU3BS9+r0Zuvn8UbJapJc2lmnW79/TC+tL5fH5w358j8+vF9aXatbv39PLm8pktUjfPn+UXvnejH45YWJnyI+ukR9AdCE/QoP86Br5ETwm4UVEGTlypCafcqp27lgd0ZMjhlPT9jWacuppGjFihNmlAAgRV4xNP54zTnNOztJtz29ScWW9bn1+k+57basWTsvVojNOUk5qaJ+eUFrdqIKP92nJ2hJVHb1impeRoMXzJ0fUiXMr8qNr5AcQfciP3iM/ukZ+BI8RMIg41y5cIPfudQp4Gs0upd8JeBrl/nydrl24wOxSAITB5NwUvfK9Gbp99lhlJblU1dCsR1bu0rm/e1ffeHKtVmypUE1jz4eX1zQ2a8WWCn3jybU653fv6pGVu1TV0KzsZJdunz1Wr34/sq5afhn50TnyA4hu5EfvkB+dIz+6x2L05wfEAx0oLS3VqNF5ck25TKnn32h2Of1K9crH5dn0mnYW71BOTo7Z5fS5xmafJtz1hiRpyz2zFedgkB+il88f0FtbK/X0R3u1euehdq/lpMZqUk6y8ocma+LQZA0fFK+AYei8B1dKklbdfr6sFov2VDWocH+tivbXanNprUqr29/fPmN0uq4/c5guGp8huy3yr9mQH50jP8gPDBzkR/eRH50b6PnRXaQLIk5OTo5+cefP9cu771ZC/kzFpPf/2dP7gvdQierXLdOv7r6bLz9gALDbrJqTn6U5+VnadbBeT3+0V+9sq9TeqkaVVjeptLpJrxV2/NSG1hPpjgwbFKcLx2Xo+jOHadTghDBVbw7yo2PkBzCwkB/dR350jPzoPkbAICK53W6Nm3CyDhqJSp9/jyxWm9klmcoI+HXo+buUYa3X1s+K5HK5zC7JFFzBBKTaJq8+21+rzftr265OVtS55fYG2q3nirEqM8ml/KHJmnT0SufJQ5OVHBtjUuV9g/xoj/xoQX4A5EdXyI/2yI+eIV0QkVwul/73r49qzpw5Ovz6H5R2yQ8G7JegEfDr8Ot/kHtfof62fDlffsAAlxwbo7NHp+vs0entlte7m5V/9wpJUtHds5TgcphRnunIjy+QHwCORX6cGPnxBfKj5yL/hjwMWLNmzVJBQYEat6zU4df/ICMQ/sfp9TetX36NW1aqoKBAs2bNMrskAP2U1Wrt8OeBiPwgPwAEj/z4AvlBfvQWI2AQ0RYuXChJWrRokQL1VUqZedOAuSfTe6hENW8/Kve+QhUUFLR9FgCArpEf5AcA9AT5QX70xsBuYSIqLFy4UMuXL9dgyxGVP/FdVa98PKofERfwNKp65eMqf+K7yrDWa/ny5Xz5AUAPkB/kBwD0BPlBfvQUI2AQFWbNmqVtWz7T4sWLde9v7lPZhlflGjFVsWOnK3bUNFkdsWaX2CuB5iY17Vqrpu1r5P58nexWq+751a906623cs8lAPQC+QEA6AnyAz3BU5AQdUpLS1VQUKBnlyzVxk/XyxbjlDNjmJSUpZjUIbKnDpEtIU0Wu0MWW4xksZhdcgvDkOH3yvA1y19/WL7qMnmry6S6cnkq98rv9WjKqafp2oULtGjRIh711gGeYgF0jt+PrpEfAxe/H0Dn+P3oGvmBYNGAQVTbvXu3XnrpJRUVFWn7jmIVF+9UZcUBs8sKSkZmtvLyRmvsmDzl5+friiuu0MiRI80uq1/jBAHoHL8f3UN+DCz8fgCd4/eje8gPnAi/PYhqI0eO1C233NJuWUNDgyorK+V2u+V2u9VfepAWi0Uul0sul0sZGRmKj483uyQAGLDIDwBAT5AfOBEaMBhw4uPjNWLECLPLAABEGPIDANAT5Ada8RQkAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDO72QUAfa2+vl6VlZVyu91yu91ml9OOy+WSy+VSRkaGEhISzC4HAHAM8gMAAPQGDRhEtd27d2vZsmUqKirS9h3FKi7eqYOV5WaXFZTBGVnKyxutsWPylJ+fr3nz5mnkyJFmlwUAAwL5AQAAQo0GDKJOaWmpnnnmGT27ZKk2bfhUthinnBnDpaQsxYy+QINOHyJbfKosdqcsNrtksZhdcgvDkOH3yfB55G+oVvPhMm08XKaNb36gfzxdoFtvvVWTTzlV1y5coOuuu045OTlmVwwAUYX8AAAA4UQDBlHD7XbrwQcf1L2/uU9+w5Br5FSlf+XHih01TVaHy+zyeiXQ3KSmXeu0c8dq/fyuX+quX96tX9z5c912221yuSL7vQGA2cgPAADQF2jAICqsWLFC3/zWTdq3b68Sp16p5LMWyOqMM7uskLE6YhU//hzFjz9HAU+jaj9cql/efbce+7/H9b9/fVSzZs0yu0QAiEjkB/kBAEBf4SlIiHhLlizRnDlzdNBIVPaN/6PU82+MqpPnL7M645R6/o3KuvF/dNBI1Jw5c7RkyRKzywKAiEN+kB8AAPQlGjCIaEuWLNGiRYsUN+F8pc+/RzHpuWaX1Gdi0nOVPv8exU04X4sWLeIkGgC6gfwgPwAA6GvcgoSItWLFiraT57RLfiCL1WZ2SX3OYrUp7ZIfSJIWLVqktLS0AT2cPBAIdPgzAByL/CA/voz8AAD0BYthGIbZRQDd5Xa7NW7CyTpoJCp9/j0D8uT5WEbAr0PP36XBliPatuWzqJ9YsbbRq6KyWm0urVXR/loV7q9VRZ1bHl/7k2an3arMJJcmDk3WxJxkTRyarPwhyUqOizGpcsA8jc0+TbjrDUnSlntmK84xMK/BkB/tkR/kB9AV8gMIHX57EJEWL16skn37lPX1Pw34k2ep5UpmysybVPLEd7V48WLdeeedZpcUcrsO1uvpj/bq7a2V2ne4MahtPL6A9h1u1L7Djfp34YG25SelxWnm+Axdf+YwjRqcEK6SAfRD5Ed75EfHyA8AQDgwAgYRp7S0VKNG58k15TKlnn+j2eX0K9UrH5dn02vaWbxDOTk5ZpfTaz5/QG9trdBTH+3Vmp1V7V7LTYttuTI5NEUThyZr2KA4GYZ07oPvSpLeu/0CWSzS3qpGFe6vVeH+GhXur1XJ4aZ2+5k+epBuOHOYLhqfKbuNabEQvbiCSX6cCPlBfgCdIT+A0OG3BxHnmWeekd8wlHzWArNL6XeSz1qosg2vqqCgQHfccYfZ5fSY2+vX31d/rqc+3KvyOrckyWqRLhyXqWum5Wrq8FSlxDmO266x2df2c3qiQ3EOu3LT4jQjL71teU1js9btqdZza0v0zrYKrdlZpTU7q5SV5NINZw3TN84ZIaedq+JANCI/Okd+kB8AgPCjAYOI8+ySpXKOmBrVjwrtKaszTq4RU/XskqURewK9saRGtz+/ScWV9ZKkQfEOXXN6rq49/STlpPb+7zwlzqGLJmTqogmZKq1u1LOf7NNzn5SovM6tB9/YrmUb9mvx/MmanJvS62MB6F/Ij86RH10jPwAAvcV4SUSU3bt3a9OGTxU3dobZpfRbsWOna+On67V7926zS+kWt9ev+1/fpqseWaPiynqlJzj18ILJ+uCnF+r22eNCcvL8ZTmpcbp99jh98NML9fCCyUpPcKq4sl5XPrJGDyzfJrfXH/JjAjAH+dE18iN45AcAoCdowCCiLFu2TLYYp2JHTTO7lH4rdtQ02WKceumll8wuJWgbS2p02Z9W69FVuxQwpCumDNGKW87VVafm9MlwbqfdpqtOzdGKW87VFVOGKGBIf1m5S5f/abU2ltSE/fgAwo/86Br50X3kBwCgO2jAIKIUFRXJmTFcVkd0PyazN6yOWDkzhqmoqMjsUoLyWuEBzX/0A+08etXyrzecpj9cc4pS44+/Rz/cUuMd+sM1p+ivN5ym9ASHiivrNf/RD/T6MU/AABCZyI+ukR89R34AAIJBAwYRZfuOYikpy+wy+r+krJbPqp9burZE3y34VF6/oVkTMrXilnM1+2Tz/35nn5ylFbecp1kTMuX1G/pOwadauq7E7LIA9AL5ESTyo1fIDwDAidCAQUQpLt6pmNQhZpfR78WkDlFx8U6zyzihpetKdMcLmxUwpIVTc/Xo9aeZctWyM6nxDj16/WlaODVXAUO645+bOYkGIhj5ERzyo/fIDwBAZ2jAIGLU19frYGW57GmcQHfFnjpElRUH1NDQYHYpHXq98IB+8sJmSdLXpw/X/VdPlM1qMbmq49msFt1/9UR9ffpwSdJPXtis5UUMJwciDfkRPPIjNMgPAEBHaMAgYlRWVkqSbPGpJlfS/9kS0iR98Zn1JxtLavT95zYoYEjXTMvVXZdNkMXS/06eW1ksFt112YS2K5nfe3aDNjGxIhBRyI/gkR+hQ34AAL7MbnYBQLDcbrckyWJ39tkxfbUV2v/of7VfaLXJFpcs59AJSjrjajmz81S39iVVv/O/is+/SOlzf9jhvtz7Nqvi2Z/LkTVaWTcslsUavqczWOwtQ7FbP7P+wu3167bnN7Xds/+bKyf265PnVhaLRfddNVGHG5u1YkuFbnt+k1753gy5YsL/hA0AvUd+BI/8CC3yAwBwLEbAIGK0nUDb+r5vaE/JVvL0a5U8/VolTb1C9rQcNW5frfKnb5e7pEiJU78i50kT1VD0lhp3fnzc9oHmJh167Q+y2GOUPvdHYT15liSLLUZS/zuB/sPbxW1Pq/jd1ZP65bDxztisFj1w9aS2p1v88e3+P0klgBbkR/DIj9AjPwAArWjAIPKYcMXLnpqtlBnXKWXGdUq94D+Vde19SjnvP6SATzXvPy2LxaL0S2+RxRGnquV/kr+prt321e/8Xf7aCqWc+x+KSc8Nf8FHPyPDMMJ/rCBtLKnRX1ftkiT95sr8fjVhYrDS4h26d95ESdKjq3YxlByINORH18iPsCA/AAASDRigxxImXSxJai5veVqEPTlDaTO/qUBDjQ6/8Ujbek2716t+03I5T5qkxKlfMaVWs7m9ft3+/CYFDOmKKUP6xaNCe2pOfpa+MnmIAoZ02/Ob5PH5zS4JQIQhP4JHfgAAogkNGKC3jhkOnjBplmJHn6HG7avVsGWVAu56VS3/kyyOOKVf+sOIuF89HP6++nMVHx06fvflJ5tdTq/96isnKz3BqeLKej32/udmlwMgUpEfXSI/AADRhAYM0EP1m96QJLlyJrRbPmjOd2WNTdLhFX/Rodf+W/4jh5Q285uyJ2eYUabpfP6AnvpwryTpZ5eOi8ih41+WGu/QTy8ZJ0l6+qO98vkDJlcEIJKQH8EhPwAA0YanIAFB8FUfUM3qZyRJhtcjT/lOefZtljU+Rann/2e7dW3xqRo0+7s6uOw+NRV/pNjRZyhh0iwzyu4X3tpaofI6t9ITHJo7KdvsckLmssnZuu+1rTpQ69ZbWys1Jz9yh8UDCB/yo+fIDwBAtGEEDBAEX80B1a55VrVrnlXdJy/Ks2+zbPGpyrrudx1Oihg39mw5ssdIklLPu7GPq+1fnvqo5erlwmm5ctqj57GbTrtNC6e1/N0/ffQ9AsCXkR89R34AAKINI2CAILhGnKrMBfdIkvyNtaovfFs1q57QwRd+rayvPSyrI/a4bSz2lqHSlpjIHzLdUzsr67VmZ5WsFuna008yu5yQW3TGSfrLql1avfOQdh2s16jBCWaX1C8cqG3S+8WHVLS/VhV1LY+yzUxyKX9oss7JS1d28vG/L0C0Ij96hvwYmMgPANGOBgzQTba4ZCWfcZUMT4NqP1yimveeUtpF/8/ssvqlZz5uubJ34bhM5aTGmVxN6OWkxmnmuAy9tbVST3+0V7+Mggkie2NbeZ0eenOH3t5aoYAh2a0W+QItj7Jt/dlqkWaOz9StF4/RuKwkkysG+hb5ETzyY2AhPwAMFNyCBPRQ0lkLZEtI05ENr8lXW2F2Of3S21srJUnXTDt+mH20WDit5crsO9sqTa7EPIGAoT+/u1Nz/7ha72yr1NFz5raT52N/Dhgtn9XcP67Wn9/dqcAx6wADBfnRNfJjYCA/AAw0Ud+Aefrpp/Wtb31LU6dOldPplMVi0RNPPGF2WYgC1hinks74qhTwqWbNc2aX0+/UNnq173CjJGnq8FSTqwmfqcNa3tveqkbVNnlNrqbvBQKGfvJioR58Y7v8AUP+IE6IW9d78I3t+smLhZxEY8AhP06M/BgYyA8AA1HUN2DuvPNO/e1vf9PevXuVnR09M+ijf0icMke2hDQ1fPaOvNUHzC6nXykqq5Uk5abFKiUueucxSI13KCe15Z70z/bXmlxN3/vLql1auq6kx9svXVeiv6zaFcKKgMhAfnSO/BgYyA8AA1HUN2Aee+wx7dmzRwcPHtRNN91kdjmIMha7Q0lnzpcCftWuKTC7nH5lc2nLyeTEockmVxJ+k3Ja3uPmAXYCva28Tg+v2NHr/Ty8Yoe2ldeFoCIgcpAfnSM/oh/5AWCgivpJeC+66CKzS0AEsydnatiPXz3hOkmnXa6k0y4/bnnWovvDVVZEKNrfegKdYm4hfSB/aLJeKyxX4QA7gX7ozd6fPLd6+M0d+tvXpoZsf4DZyI+eIz+iH/kBYKCK+hEwAMxRuH/gXMFsfY9FA+gE+kBtk97aWhHUPftd8QcMvbW1Qgdqm0JQGYBIR35EN/IDwEBGAwZAWFTUuSVJwwZF3+NDv2z4oHhJX7zngeD94kMyQjj3YcCQVhcfCt0OAUQs8iO6kR8ABrKovwUp2hiGoSav3+wyTNHk9ZldQsRp8vrU2Nz3n1sgEJDHF5AkGYb6rIZjj9OX7ztw9EzS7Q2o3t0sqzX6e9sb91XLbrW0e1Rob9itFm0oqdHcSUyWHi5m/X70B+RH95EffYP86D3yI/wGcn70d7ExNlksFrPLQDfQgIkwTV6/Jtz1htllmMJTvtPsEiLOV//yoZxZB02t4dwH3zXluFPvfduU4+bfvcKU40Y6X8BQwcf7VPDxPrNLGRDM+v0wC/nRfeRH3yM/eob86FsDLT/6uy33zFacg//SR5Lob7MDAAAAAACYjHZZhImNsWnLPbPNLsMUGzYM1ownza4isvzz5rN0yimn9vlxA4FA25W8926/QOmJjj45bmOzr+3KzLo7Z/bZFYGDRzw678GVkqSiu2cNiCHk9766RUvXlYZ0CPmCabm6c+74kOwPxzPr96M/ID+6j/wgP8KF/Ig8Azk/+rvYGJvZJaCb+O2JMBaLZcB+6cXGDMz33RuxMXbT/r047VZ5fAFZLDKlhjhH3713q6VZkuSKsSrB1Tf/WTDblJNSVfBJScj25wsYOiU3ZcB+v/W1vvz96A/Ij+4jP8iPcCE/IttAyw8g1KK/zY4BzTAMHXjiB6pY8guzS+mRpj0btfeBy9S0a63ZpXRbZpJLkrS3qtHkSsJvT1WDpC/e80BwTl66rCGc881qkc7JGxy6HQK9FOn5cWTTG9r7u6+o+eAes0vpNvIjupEfAAayqG9fPvbYY1q9erUkqbCwsG3ZypUrJUkzZszQN77xDbPKQ5g1FL2j5opdyrp+cbvlhs+r2o//qYbP3pGv7pBssYmKHTVNKefcIFt8StD7N4yAjnz6bzUUviVvValktcqRMVJJp1+luLwzjlu/ZvUzql3zbKf7G3rT32VPzmz7c+zwKXLmTFD1ysflGnGqLNbIGWY4cWiy9h1uVOH+Ws3ISze7nLAq3F8rScofmmxyJX0nOzlWM8dn6p1tlfL3chi5zWrRzHEZykoeOP8BQf8X7vyo/+xdeUo+U3PFzpYmid+nQZf+UAkTLzrhdt6actV9uFRNezbI31AtqzNBMem5SjxlruLHzWhbLyF/puo+XKrqd/9PmQvu6c5bNx35Ed3IDwADWdQ3YFavXq0nn2x/4/eaNWu0Zs2atj/TgIlOhhFQzZoCOXNOlnPouHbLK1/8tdyffyrHkLFKGjNdvuoy1W96U+69m5R1w0OyxXV9ImQYhg4tu1+NOz6QPSVbCZNmyfB71Vj8sQ6++GulXvQtJZ12eYfbxufPlD0547jlVmf8ccuSzrhaB1/4tRq2vqeEky/oxidgrvyhyfp34QEV7q8xu5SwKzp6Aj1xAJ1AS9KtF4/RO9sqQ7KvH108JiT7AUIh3PkhSTXvPSV/XaWssUmyxafJX9f171LT5xt08F/3SpJiR50ue0qWAu56eQ/ukXvPxnYNGIvNrsRp81T91l/lLt0iV86Ebn4K5iE/oh/5AWCgivoGzBNPPKEnnnjC7DJggqbd6+WvrVDyWQvaLW8ofFvuzz9V3PjzlH75bbJYWsbBHtnwmg6/+Yhq3ntKg+Z8t8v9N25fo8YdH8g5dIIyFv5a1hinJCnl3K/pwJO3qPrd/1Pc6NPbjWhplTBxplwnTQrqfcSOOE3W2CTVb3w9ohowk3JaTiZbr+5Fs82lLe9x0gA7gR6XlaQfzRqjB9/Y3qv9/GjWGI3LSgpRVUDvhTs/JGnQJd9XTOoQ2ZMzVPvR86pZdeJZgn11lTq47LeyJQxS5jX3yp7UvolvBPzHbRM//lxVv/OY6je+HlENGPIj+pEfAAYq5oBB1GrYvEKSRXFjp7dbXr/pDUlS6nn/0XbyLEkJUy6RPSVLDVtWKuD1dLn/pp0fS5KSz5rf1nyRJFtcspKmXSH5varf/Fav34fFZldc3pnylG6Rt7qs1/vrK/lDWk4mSw43qaax2eRqwqe6oVml1U2SpJMH2Am0JN183igtmJrb4+0XTM3Vt88fFcKKgN4Ld35ILbeYdjQSsjO1Hy6V0dyotNnfOa75IqnDW1Rtccly5U5U4/Y1CjQ3BX0ss5EfAwP5AWAgogGDqGQYhtz7CmVPGyqbK+GL5b5meQ7skD0t57gTX4vFItfwKTK8bjWXF3d5DH99tSTJnpx13Guty9z7NnW4rbvkM9V+9E/VfvyCGnd82OWJcesQePfezV3W1V8kx8XopLQ4SdK6PdUmVxM+6/a2vLdhg+KUHBtjcjV9z2q16P6rJur22WNls1pkC2Jmxdb1bp89VvdfNbHdf2QBs/VFfvSkpsZta2SNTVLssMnylO9U3Sf/Uu3HL6ppz0YZRqDTbZ1Dx7XUvn9ryOsKF/JjYCA/AAxEUX8LEgYmb1WJAu4jih15Wvvl1QckI6CY1OwOt4tJHSJJ8lWXSbn5JzyGNa5lyKuvtlwx6e2v4Phqy1uOd7jjESu1q59pvy9nvFIv+n9KyJ/Z4fqOrDxJkmf/FiVOmXPCuvqTmeMz9PiaPXpubYkumnD8rVjRYMnafZKkC8cFfyU72litFn3ngtGaOT5DD7+5Q29trVDAkOxWi3xHJ1hs/dlqkWaOy9CPLmbYOPqnvsiP7vLVVijgPiJHVp6qlv+P6jctb/e6I3OUBl/1C9mTjp+wti0/SrcqdsSpIa0rnMiPgYH8ADDQ0IBBVPIfOSRJsn7piRQBT8sjLTua7FaSLI64duudSOzI09S49T3VfvRPuYZNlsXuaDl2U53q1r18dD/17bZxZIzQoEt+IOdJE2VPSJO/oVqNOz9R7epnVPXv/5bVmdDh05Nan6zR+r4ixfVnDtPja/bonW0VKq1uVE5qnNklhVTJ4Ua9fXQSwevPHGZyNeYbl5Wkv31tqg7UNml18SFtKKlRwcct/8FYMC1Xp+SmaEZeurKTY02uFOhcX+RHdwUaaiRJzRW75D1cqkGX/lCxeWfK8DSo9sOlqt/0hg4u+62yv/bQcdu25oeP/OhXyI/2yA8AAwW3ICEqBZqOSOr8RDkU4iecL+dJk+Qp/Uxl//cdHV7xqKre+B+V/f3bsjpaThAslva/YnFjzlbCpFmKScmSxe6QPTlTSaddrvQrfiJJqnn/qQ6PZXUlSpL8jXVhez/hMGpwgqaPHqSAIT37yT6zywm5Zz/ZJ8OQZoxO16jBCV1vMEBkJ8dq/tRc3Tl3fNuyO+eO1/ypuZw8o9/ri/zoLsM4+qheI6CUGdcrYeJFsrkSZE/O1KA535Mje6yaD2yXu/Sz47a1Hr2NKtBEfvQn5EfHyA8A0Y4GDKJS62gUw+9tt9zqbL1C2dDhdkZzY7v1TngMq02Z83+l5OmLJItVRzYtV+OODxU3+kwNnvfTlv0E+TjS2OFTZE/Nkvfgng6vnhq+lkkILcdM9hspbjh6ZW/J2hJ5fMc/pSNSeXx+LVlbIomrl0A06Yv86K5j9xnbwSjJuNGnS5KaD+w8vi7yo98hPwBg4OIWJESl1sZH65XMVvaULMlibbmXvwOtTxmyH72XvysWe4xSZixSyoxF7Za797VMlus8eu99UDXHJknVB2R4PdKXTuAD7pb3YQuyodOfXDQ+U1lJLpXXufXvzQd01ak5ZpcUEq9uOqCqhmZlJ7t00fiBe/8+EG36Kj+6IyY1W7JYJSPQ4cgcq6tlmeE7/glM5Ef/Q34AwMDFCBhEJUf6SS0nyof3t1tujXHKkZ0n3+FS+Wor271mGIbcezbKEuNqm7Swpxo+WylJiht/blDrB5rd8h7aJ0uMq21y32N5q0olSTGDh/eqLjPYbVbdcFbLFb77Xtum6obIf6RodUOzfvv6NkktVy/tNr5KgWhhdn50xGJ3yDm05ZYMb9Xxt+M0H2oZTWFPPn6y2tb3EZMeeSMtyA8AQLThWx9RyepKUMzg4WouLz7u8ZyJk1ueIlS96skv7quXVL/xdflqyhU/4XxZjxmqbfh98laVdHjVs6PbhRq2rVZ94VtyZOcpbsxZ7db98gm9JAW8HlUt/5OM5ibFjZshi9V23DqeAzskSa4QP1mjr3zjnBHKy0jQoXqP7n7l+DkKIs0vX/5Mh+o9ystI0DfOGWF2OQBCqK/yo7sST7lUklS7ukCG74vbo7xVJWooeksWR6xcX3pykyR5yrZLklwnTex1DWYgPwAA0YRbkBC14sacpdrVz8izf7tcOV9M5hY/caYatr2vxq2rVF5bLlfuRPmqy9S440PZkzOVcu4N7fbjr69S2WM3y5aUoZyb/6/dawf+8SPZkwYrZlCOLHaHPAd2yLOvUPaULA2+4qftmikB9xGV/e9NcmTnKWZQrmzxqfI31Mi9d6P8Rw4pZvBwpV7wnx2+F/eeDbK6EiK2AeO02/Tg/Mm66pE1emljmS6dmK3ZJ2eZXVaPLC8q18ubymSzWrR4/mQ57cc3zABEtr7IjyOb3pCndIskyXtwjySpftObcu8rlCQ5cyYocfLsL2oaf67idnygxu1rVPb49xQ74hQFPI1q3PGBDJ9Xg+beIpur/WSuhmHIvXeTYgblKiZtaMg+n75EfgAAogkjYBB5jrnqeCIJky6WrDY1bHm33XKLxaqMq36h5OmLFGisU926ZfLs36qESbOUdcPibt0nHz/+HPkbqlVf+JaOrH9FgYYaJZ+1UNk3/lH25Pb3dVtdiUo89VJJhpp2r1Pd2n+pccca2RLSlHL+15V1w0OyxR5/+5GvtkKe0q2Kz5/ZNjlkl45+RhaLJej3Em5TclP0rfNGSZJ+/q+iiBxKfrihWXcua/nP0bfOHanJuSnmFgSge/pRfnhKt6ih6G01FL2t5opdLcv2f7GstTnzxbEtSv/KHUq98BuyWK06srFl4nfnkHHKvOY3Sjj5guOPUVIkf91BJUyZE3Rd5Ed4kB8AAIkRMIggLpdLUsuQ7mDYEwcpfvy5atz6nlLPu7HdUyQ6mzy3w/0kZ2rYj1/t8LWUGdcpZcZ1QdVjdcYpbdbNQa17rCOb3pSsNiWeelnQ27Q+vaP1M+svfjAzT29tqVBxZb3ueGGzHr3+NNms/eck/0T8AUM/fmGzDtU3Ky8jQT+4KPTzPAAIj/6YH+lzb1H63FuCqqft2FabkqbNU9K0eUGtX7/pDVldiUrInxn0MciP0CM/AACtGAGDiNF2At3BUx46k3LODTJ8zTryaccnwP2d312vI+tfUeIpl7Q8BSNIrY8d7W8n0K6YlqHkMTaLVmyp0M9eLGw3j0J/ZRiGfvZioVZsqZDDZmXoOBBhBmJ+eA/vV8PW95R89kJZv3Rr0omQH6FFfgAAjkUDBhEjI6Pllh5/Q3XQ29iTMzTo0ltkccSGq6yw8tWUK2naFUqefm23tvPXH5b0xWfWn0zJTdEfrzlFVou0ZF2J7nl1S78+iTYMQ/e8ukVL1pXIapH+eO0Uho4DEWZA5seRQ0qefq0ST53bre3Ij9AhPwAAX0YDBhEjISFBgzOy5Dtc1q3t4sefo6TTLg9TVeHlzBqtlBnXdTg3zIn4qsuUkZmt+Pj4MFXWO5dMzNb9V0+SJD2+Zo9+8kKh/IH+dxLdOmz88TV7JEkPXD1Jc/KDH4kEoH8YiPkRO2yyUqZfK4stplvbkR+hQX4AADpCAwYRJS9vtLzV3TuBHoi81WXKyxttdhkntGBqrn731UltVzJvenp9v5pY8XBDs256er2WriuV1SI9+NVJmj811+yyAPQQ+REc8qP3yA8AQGdowCCijB2TJ9WVm11G/1dX3vJZ9XMLpubqz4tObbunf9bvV2l5kfl/v8uLynXx71dpxZYKxdgseuS6Uzl5BiIc+REk8qNXyA8AwInQgEFEyc/Pl6dyjwLNTWaX0m8Fmpvkqdyr/Px8s0sJyiUTs/X8TWdrdEaCDtW3XDX8wXMbTLmaWd3QrO8/u0E3Pb2+7WkV/7zpbIaNA1GA/Oga+dFz5AcAIBg0YBBR5s2bJ7/Xo6Zd68wupd9q2rVWfq9H8+bNM7uUoE3JTdGr35uhm88fJatFemljmWb9/j29sL5UHp8/7Mf3+Px6YX2pZv3+Pb28qUxWi/Tt80fple/NYMJEIEqQH10jP7qP/AAAdIfd7AKA7hg5cqQmn3Kqdu5Yrfjx55hdTr/UtH2Nppx6mkaMGGF2Kd3iirHpx3PGac7JWbrt+U0qrqzXrc9v0n2vbdXCabladMZJykmNC+kxS6sbVfDxPi1ZW6Kqo1dM8zIStHj+ZE6cgShDfnSN/Age+QEA6AkaMIg41y5coJ/f9UsFPI2yOkN7QhXpAp5GuT9fp2t/fY/ZpfTY5NwUvfK9Gfr76s/11Id7VV7n1iMrd+nRVbt04bgMLZx2kqYNT1VKnKNH+69pbNbaPdVasnaf3t5WqdYnmGYnu3T9mcP0jXNGyGm3hfAdAegvyI/OkR9dIz8AAL1FAwYR57rrrtNdv7xbtR8uVer5N5pdTr9S++ES2a1WLVq0yOxSesUVY9N3Lhitb507Um9trdTTH+3V6p2H9NbWSr21tVKSlJMaq0k5ycofmqyJQ5M1fFC8AsYXjyI9eMQjq6VZe6oaVLi/VkX7a7W5tFal1e3nf5gxOl3XnzlMF43PkN3GXZlANCM/Okd+kB8AgPCjAYOIk5OTo1/c+XP98u67lZA/UzHpPF1AkryHSlS/bpl+dffdysnJMbuckLDbrJqTn6U5+VnadbBeT3+0V+9sq9TeqkaVVjeptLpJrxV2/NSL8x5c2el+hw2K04XjMnT9mcM0anBCmKoH0N+QHx0jP9ojPwAA4WIxjGNa/kCEcLvdGjfhZB00EpU+/x5ZrAN7yK8R8OvQ83cpw1qvrZ8VyeVymV1SWNU2efXZ/lpt3l/bdnWyos4ttzfQbj1XjFWZSS7lD03WpKNXOk8emqzk2BiTKh9YGpt9mnDXG5KkLffMVpyDnr+Z+PtoQX60R36QH/0R31f9C38fQOjw24OI5HK59L9/fVRz5szR4df/oLRLfjBgT6KNgF+HX/+D3PsK9bfly6P+5FmSkmNjdPbodJ09Or3d8kAgoFq3t2UdV4ysVoaEA2iP/PgC+fEF8gMA0BdIF0SsWbNmqaCgQI1bVurw63+QEQj/4yb7m9aT58YtK1VQUKBZs2aZXZKprFarUuOcSo1zcvIMoFPkB/nxZeQHAKAvMAIGEW3hwoWSpEWLFilQX6WUmTcNmHv6vYdKVPP2o3LvK1RBQUHbZwEA6Br5QX4AANDXaPEj4i1cuFDLly/XYMsRlT/xXVWvfFwBT6PZZYVNwNOo6pWPq/yJ7yrDWq/ly5dz8gwAPUB+kB8AAPQlRsAgKsyaNUvbtnymxYsX697f3KeyDa/KNWKqYsdOV+yoabI6Ys0usVcCzU1q2rVWTdvXyP35OtmtVt3zq1/p1ltvHRD37ANAuJAfAACgr9CAQdRwuVy68847deONN6qgoEDPLlmqjS//TrYYp5wZw6SkLMWkDpE9dYhsCWmy2B2y2GIki8Xs0lsYhgy/V4avWf76w/JVl8lbXSbVlctTuVd+r0dTTj1N1/76Hi1atChqHhUKAGYjPwAAQF+gAYOok5OTozvuuEN33HGHdu/erZdeeklFRUXavqNYxcWrVFlxwOwSg5KRma38vNEae9YM5effpCuuuEIjR440uywAiFrkBwAACCcaMIhqI0eO1C233NJuWUNDgyorK+V2u+V2u2UYhknVtWexWORyueRyuZSRkaH4+HizSwKAAYv8AAAAoUYDBgNOfHy8RowYYXYZAIAIQ34AAIDe4ClIAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGZ2swsA+lp9fb0qKyvldrvldrvNLqcdl8sll8uljIwMJSQkmF0OAOAY5AcAoCfID7SiAYOotnv3bi1btkxFRUXavqNYxcU7dbCy3OyygjI4I0t5eaM1dkye8vPzNW/ePI0cOdLssgBgQCA/AAA9QX7gRCyGYRhmFwGEUmlpqZ555hk9u2SpNm34VLYYp5wZw6WkLMWkDpE9bYhs8amy2J2y2OySxWJ2yS0MQ4bfJ8Pnkb+hWr7DZfJWl0l15fJU7pHf69HkU07VtQsX6LrrrlNOTo7ZFQMn1Njs04S73pAkbblntuIc9PzNxN9H18gPoH/g+6p/4e+ja+QHgkUDBlHD7XbrwQcf1L2/uU9+w5Br5FTFjpmh2FHTZHW4zC6vVwLNTWratU5NO1bLvXudbBaLfnHnz3XbbbfJ5Yrs94boxQlb/8LfR+fID6B/4fuqf+Hvo3PkB7qL3x5EhRUrVuib37pJ+/btVeLUK5V81gJZnXFmlxUyVkes4sefo/jx5yjgaVTth0v1y7vv1mP/97j+96+PatasWWaXCAARifwgPwCgJ8gP8qMneAoSIt6SJUs0Z84cHTQSlX3j/yj1/Buj6svvy6zOOKWef6OybvwfHTQSNWfOHC1ZssTssgAg4pAf5AcA9AT5QX70FA0YRLQlS5Zo0aJFiptwvtLn36OY9FyzS+ozMem5Sp9/j+ImnK9FixbxJQgA3UB+kB8A0BPkB/nRG9yChIi1YsWKti+/tEt+IIvVZnZJfc5itSntkh9IkhYtWqS0tDSGA6LfCAQCHf4MmI38ID/Qv5Ef6K+iKT989YdlsTtkc3Xv0dPkR+8wCS8iktvt1rgJJ+ugkaj0+fdE9JdfKBgBvw49f5cGW45o25bPmBgLfaq20auislptLq1V0f5aFe6vVUWdWx5f+5Nmp92qzCSXJg5N1sScZE0cmqz8IclKjosxqfKBhUkUW5Af7ZEfMBP5ERnIjxbRlB/NlbtV+fyvJElDv/2ELD14KhP50TMD87cHEW/x4sUq2bdPWV//U0R/+YWKxWpTysybVPLEd7V48WLdeeedZpeEKLfrYL2e/miv3t5aqX2HG4PaxuMLaN/hRu073Kh/Fx5oW35SWpxmjs/Q9WcO06jB3bsKA3QX+dEe+YG+Rn4gUkVLfjTtWquDL/9ORnOTHNlje9R8kciPnmIEDCJOaWmpRo3Ok2vKZUo9/0azy+lXqlc+Ls+m17SzeIdycnLMLgdRxucP6K2tFXrqo71as7Oq3Wu5abEtVyaHpmji0GQNGxQnw5DOffBdSdJ7t18gi0XaW9Wowv21Ktxfo8L9tSo53NRuP9NHD9INZw7TReMzZbcxTVkocQWT/DgR8gPhRH5ENvIjevKjbv0rqn77fxU7apo8B7Yrfvx5Spv5zV7tk/zonoH324OI98wzz8hvGEo+a4HZpfQ7yWctVNmGV1VQUKA77rjD7HIQJdxev/6++nM99eFelde5JUlWi3ThuExdMy1XU4enKiXOcdx2jc2+tp/TEx2Kc9iVmxanGXnpbctrGpu1bk+1nltbone2VWjNziqt2VmlrCSXbjhrmL5xzgg57ZF7lQn9C/nROfID4UB+IFpEen4YAb+q3/m7jqx/WYlTr1DS6Vdp/yP/IeeQcb3eN/nRPTRgEHGeXbJUzhFTo/pRbz1ldcbJNWKqnl2ylC9AhMTGkhrd/vwmFVfWS5IGxTt0zem5uvb0k5ST2vvfwZQ4hy6akKmLJmSqtLpRz36yT899UqLyOrcefGO7lm3Yr8XzJ2tybkqvjwWQH50jPxBq5AeiSSTnR6C5SYdeeVBNu9YpbdbNSjx1rhp3fChJcg4d2+v9kx/dw/g8RJTdu3dr04ZPFTd2html9FuxY6dr46frtXv3brNLQQRze/26//VtuuqRNSqurFd6glMPL5isD356oW6fPS4kJ89flpMap9tnj9MHP71QDy+YrPQEp4or63XlI2v0wPJtcnv9IT8mBg7yo2vkB0KB/EC0ieT88B2pUkXBT+TeV6iMq+9S4qlzJUmesm2yJaTJljg4JMchP4JHAwYRZdmyZbLFOBU7aprZpfRbsaOmyRbj1EsvvWR2KYhQG0tqdNmfVuvRVbsUMKQrpgzRilvO1VWn5vTJcG6n3aarTs3RilvO1RVThihgSH9ZuUuX/2m1NpbUhP34iE7kR9fID/QW+YFoFKn50Vy5W+VP3Sp/Y62yrntAsaOmtr3m2b9NziHjejwB75eRH8GjAYOIUlRUJGfGcFkdPOasM1ZHrJwZw1RUVGR2KYhArxUe0PxHP9DOo1ct/3rDafrDNacoNf74e/TDLTXeoT9cc4r+esNpSk9wqLiyXvMf/UCvH/MEDCBY5EfXyA/0BvmBaBWJ+dG0a63Kn/mxbHHJyrrhITkyRra9Zvh9ai7fKefQ3s//0or8CB4NGESU7TuKpaQss8vo/5KyWj4roBuWri3Rdws+lddvaNaETK245VzNPtn837fZJ2dpxS3nadaETHn9hr5T8KmWrisxuyxEGPIjSOQHeoD8QDSLtPw48umrqnzh13KdNEmZix6QPXFQu9ebD+6R4fPIMWR8aA9MfgSFBgwiSnHxTsWkDjG7jH4vJnWIiot3ml0GIsjSdSW644XNChjSwqm5evT600y5atmZ1HiHHr3+NC2cmquAId3xz82cRKNbyI/gkB/oLvID0S6S8qPp8091eMWjihs7XYOv/FmHo3Y8+7dKVrucWaNCemzyIzg0YBAx6uvrdbCyXPa0yPgCNJM9dYgqKw6ooaHB7FIQAV4vPKCfvLBZkvT16cN1/9UTZbOG5p7gULJZLbr/6on6+vThkqSfvLBZy4sYTo6ukR/BIz/QHeQHol2k5Yc9JVv2lGw17vhANe/9Q4HmpuPW8ZRtkyNzlCz20DZKyY/g0IBBxKisrJQk2eJTTa6k/7MlpEn64jMDOrOxpEbff26DAoZ0zbRc3XXZhJBNyBYOFotFd102oe1K5vee3aBNTKyILpAfwSM/ECzyAwNBpOVHTGq2hvzXn5U8/VodWf+qSv/8NdV+/IIMw2hbp3n/tpDO/9KK/AiO3ewCgGC53W5JksXuDPuxfLUV2v/of7VfaLXJFpcs59AJSjrjajmz89peqi98S1Wv/XdQ+3bm5itr0f0hrPZ4rR3t1s8M6Ijb69dtz29qu2f/N1dO7Ncnz60sFovuu2qiDjc2a8WWCt32/Ca98r0ZcsWE/wkbiEzkR/DIDwSD/MBA0Zf5ESoWu0MpZ1+j+DHTVfb3m1Wz8nG5P/9UaRfdJGtsgny1FXIOCX0DhvwIDg0YRIy2L0Bb3/2ztadkK/7k8yVJhtcjT/lONW5frcbij5R5zb1y5eZLkhwZI5U8/doT7uvIp/9WoKlOMenDwl22LLYYSXwB4sT+8HZx29Mqfnf1pH45bLwzNqtFD1w9SRv2rVJxZb3++Hax7pgT+pMJRAfyI3jkB4JBfmCgMCM/QqXi+V9KkuLzL5Kn9DOVPf5duYZNliQ5h44N+fHIj+BE3r8koA+vsNhTs5Uy47p2y2o/el41q55UzftPt12JdGSOlCNzZEe7kCTVffKiAk11cmSNVtqF/9XpeiFz9DM6drghcKyNJTX666pdkqTfXJnfryZMDFZavEP3zpuom55er0dX7dLsk7M0OTfF7LLQn5EfXSM/0AXyAwOSiSO8jIBf3sP75a38XM0H9yjgrpfh87aUZY+R1ZUgx+DhiskYoZi0obJYbfKU75S/ruVWoPS5P5Tha1btJy+q7sPnZU/Jli1xcOgLJT+CQgMG6KaESRerZtWTai4Pbpbvpj0bVb3yCVnjUjT4yp+HfMIroLvcXr9uf36TAoZ0xZQh/eJRoT01Jz9LX5k8RC9vKtNtz2/Sq9+fIaedoeTon8gPRDryA+gbviNVavjsXTXuWCPvwb0yfM2SJFvCINniU9pGmxh+r/wNNaqrr5LUchtQzOBhaj7Q8jjoIf/vb23LU86+RgkTZ8nqSoiIWwajFQ0YoKesXYe0t6Zch156QLJYNHjeT2RPCkO3Geimv6/+XMVHh47fffnJZpfTa7/6ysn6YFeViivr9dj7n+s7F4w2uyTgxMgPRCjyAwgfI+BXY/FHati8Qk2ffyqLza7Y0WcofsL5cmSMUMzg4bLFJnW4rb+pTt7KPWo++LnqN73Ztrz6rb8pftIsxeWdKYvVJnvioL56O+gET0ECuql+0xuSJFfOhBOuF2h26+CL9yrgPqLUC7/Rdr8/YCafP6CnPtwrSfrZpeMicuj4l6XGO/TTS1ru33/6o73y+QMmVwR0jPxAJCM/gPDxlG1X+T9+pEPLfit/0xGlXXyzcr7zDw2+4sdKmnqFXCdN6rT5Ikm22CS5hk1SwuTZ8h5q+T1Nvehb8jfV6dCy36r8Hz+S58COvno7OAFGwAAn4Ks+oJrVz0j6YhJFz77NssanKPX8/zzhtlWv/V7eg3sUP/EiJZ12eV+UC3Tpra0VKq9zKz3BobmTss0uJ2Qum5yt+17bqgO1br21tVJz8iN3WDyiA/mBaEN+AKHnd9erZtWTqt+4XDEZI5R1/YNyDh3f4/2VPfZtSVLanO8pcfJsJZ12uTz7t6rqzb+o/B+3KuGUS5Ry7tdkcyWE6i2gm2jAACfgqzmg2jXPtltmi09V5nUPKCZ1SKfb1X6wRI3b18iRPVaDLv5OuMsEgvbURy1XRRZOy42qe92ddpsWTsvVIyt36emP9nICDdORH4g25AcQWp4DO1pGOzY3KXXmN5V46lxZgrhFtdP9HTPxbuLk2W3LnUPHK/s/fq8jn/5bNe8/pabijzT4qjvlzB7T6/eA7uMWJOAEXCNO1bAfv6phP35VOd97Rinn/6f8jbU6+MKvFWhu6nCbxl1rVbP6GdniUzX4yp/KYo/p46qBju2srNeanVWyWqRrTz/J7HJCbtEZJ8likVbvPKRdB+vNLqffOFDbpKXrSnTvq1valt376hYtXVeiA7Udf4+h98gPRBPyY2AiP8KnsfhjVRT8VLakwRryjUeVNPUrvWq+SFL5kz+U9MXEu8eyWG1KmvoVDfnGo7IlDVZFwU/VWPxxr46HnmEEDBAkW1yyks+4SoanQbUfLlHNe08p7aL/124d7+H9OvTKYslqVfq8n8qemG5StcDxnvm45erlheMylZMaZ3I1oZeTGqeZ4zL01tZKPf3RXv0yCiaI7I1t5XV66M0dentrhQKGZLd+8cSDpetKVfBJiawWaeb4TN168RiNy+r83nL0DvmBSEd+DCzkR3g1bFutQy//TnF5Z2rQZbfKGuPs9T5rjo64dA6dcMJRlvbEQcq85j5VvfqQDi67T+mX3674cTN6fXwEjxEwQDclnbVAtoQ0Hdnwmny1FW3LA55GVb7waxmeBqVd9K0uJ1kE+trbW1uGpV4zLdfkSsJn4bSWK7PvbKs0uRLzBAKG/vzuTs3942q9s61SAaNlua/1h2N+Dhgtn9XcP67Wn9/dqcAx6yD0yA9EKvJjYCA/ws9dUtTSfBl/jtKv+HFImi8Br1u1R+ccy7z2vi7Xt8Y4lX7FjxU3boYOvfw7uUuKel0DghfVDZj9+/frv//7v3XxxRfrpJNOksPhUFZWlq6++mp9/DFDrtAz1hinks74qhTwqWbNc5IkwzB06NXF8h0uVcLkOUqcconJVQLt1TZ6te9woyRp6vBUk6sJn6nDWt7b3qpG1TZ5Ta6m7wUChn7yYqEefGO7/AFD/iBOiFvXe/CN7frJi4WcRIcR+YFIRH4MDORH+PmbjujQK4vlzJmg9Lk/6vUtR62OnXjXYgvuBheL1ab0uT+Sc+h4HXrlIfmbjoSkFnQtqm9B+tOf/qQHHnhAo0aN0sUXX6zBgweruLhYy5Yt07Jly1RQUKCFCxeaXSYiUOKUOar7+J9q+OwdJZ+1QI07PlDTzk8km13W2MS2J190JmXGdX1UKdCiqKxWkpSbFquUuMh/dGhnUuMdykmNVWl1kz7bX6uzRw+s2zj+smqXlq4r6fH2S9eVaNigOH3ngtEhrArHIj8QaciPgYH8CC/DMFS1/I8yvG6lX3ZryJovnU28GwyL1ab0y2/Vgce/r8PL/6T0eT+VxWLpekP0SlQ3YE4//XStXLlS5513Xrvl77//vmbOnKmbb75Z8+bNk9PZ+6FfGFgsdoeSzpyv6rf+qto1BZLl6GAyv091Hz3f5facQKOvbS5tOYGeODTZ5ErCb1JOskqrm7R5gJ1Abyuv08MrdvR6Pw+v2KGZ4zO4pz9MyA9EGvIj+pEf4Ve/eYWadnyowVf+TPakwSHb74km3g2GPSlDg+Z8XweX3af6zSuUOPnikNWGjkV1A+aqq67qcPk555yjCy64QG+++aYKCws1derUPq4M/Z09OVPDfvzqCddJOu1yJZ12eduf0+feEu6ygB4r2t96Ap1ibiF9IH9osl4rLFfh0fc8UDz0Zu9Pnls9/OYO/e1rZGNPkB+INuRH9CM/wssI+FX7wXOKG3+u4sacHbL9Bjvxblfixp6tuHHnqPaD55QwcWbIRuegY1E9B8yJxMS0PNrRbo/qHhQASFLbyeRAuILZ+h6LBtAJ9IHaJr21tSKoe/a74g8YemtrBY8YBSCJ/Ih25Ef4NW5fI39dpZLP+GrI9tndiXe7knzmV+Wvq1Tj9g96vS+c2IBswOzbt09vvfWWsrOzNXHiRLPLAYCwq6hzS5KGDYq+x4d+2fBB8ZK+eM8DwfvFh2SEcO7DgCGtLj4Uuh0CiFjkR3QjP8LLMAzVffIvuYZNkSNzZMj225OJd0/EkTlKrmGTVbf2RRmh/AeB4wy44R9er1c33HCDPB6PHnjgAdlskTXEyjAMNXn9Zpdhiiavz+wSIk6T16fGZj63gS4QCMjjC0iSDEN99m/i2OP05b/DwNETB7c3oHp3s6zW6L/WsHFftexWS7tHhfaG3WrRhpIazZ2UHZL9mY386D7yAxL5QX50H/nRXvOBHWouL1bG/F+FqKLeTbx7IknTrlTlP+9Wc3mxnNljQrZftDegGjCBQEA33nij3nvvPX3zm9/UDTfcYHZJ3dbk9WvCXW+YXYYpPOU7zS4h4nz1Lx/KmXXQ7DLQj5z74LumHHfqvW+bctz8u1eYctxI5wsYKvh4nwo+3md2KSFBfnQf+YEvIz8QDPKjPXfJZ7LEOOUaPiU0Ban3E+92xjXiFFnsTnlKimjAhFH0t3WPCgQC+s///E8VFBTo+uuv16OPPmp2SQAAAACAKNVctk2OrLyQTWwbqol3O2Kx2uTIzpOnbHtI94v2BsQImEAgoK9//ev6xz/+oWuvvVZPPPFExA4pjI2xacs9oRtqFkk2bBisGU+aXUVk+efNZ+mUU041uwyYLBAItF3Je+/2C5Se6OiT4zY2+9quXK67c6biHH0TOQePeHTegyslSUV3z4rY7/vuuPfVLVq6rjSkQ8gXTMvVnXPHh2R/ZiM/uo/8gER+kB/dR3605ynbrviTLwhJLaGeeLcjziFj1bBlVVj2jRZR34A5tvmycOFCPfXUUxE378uxLBZLn4VQfxMbMzDfd2/ExtgH7L8XtOe0W+XxBWSxyJR/E3GOvvu3aLU0S5JcMVYluPrmPwtmm3JSqgo+KQnZ/nwBQ6fkpkTN9wf50X3kB1qRH9GN/Dix3uSHv6Fa/voqObLzQlJL2d+/Iyl0E+92xJE9RnUfvyB/Q7Vs8alhOcZAF9Vt3dbbjv7xj39o/vz5evrppyO6+YLuMwxDB574gSqW/MLsUnrkyKY3tPd3X1HzwT1ml4IIl5nkkiTtrWo0uZLw21PVIOmL9zwQnJOXLqsldPuzWqRz8gaHbocRiPwAWpAf0Y38CJ+Au16SZIvr/SPcPeU75a+tkBTaiXe/rLXWgLshbMcY6KKjNdmJe+65R08++aQSEhI0ZswY3XvvvcetM2/ePE2ZMqXvi0OfaCh6R80Vu5R1/eJ2yw2fV7Uf/1MNn70jX90h2WITFTtqmlLOuUG2+JSg9l1f+JaqXvvvE67jGjZJmdd8MUSw/rN31bh9jbwH98jfWCsZhuxJGXKNOEVJp18pe2J6u+0T8meq7sOlqn73/5S54J6g6gI6MnFosvYdblTh/lrNyEvveoMIVri/VpKUP7T3JzyRIjs5VjPHZ+qdbZXy93IYuc1q0cxxGcpKHjj/AelIOPNDaskDT8lnaq7Y2dIk8fs06NIfKmHiRSfczltTrroPl6ppzwb5G6pldSYoJj1XiafMVfy4GW3rkR8IFfIjupEf4WOxOxSbd2ZIRpKEa+LdL7PFpyo270xZ7ANjBJgZoroBs2fPHklSfX29fvOb33S4zvDhw2nARCnDCKhmTYGcOSfLOXRcu+WVL/5a7s8/lWPIWCWNmS5fdZnqN70p995NyrrhoaA61Y6MkUqefm2HrzVuXyPvoX1yjWh//3zj1vfkrT4gx5CxssenyZAhb+VuHVn3shoK31Lmdb+TY/CwtvUtNrsSp81T9Vt/lbt0i1w5E3r4aWCgyx+arH8XHlDh/hqzSwm7oqMn0BMH0Am0JN168Ri9s60yJPv60cUD++kH4c4PSap57yn56ypljU2SLT6t7ZGiJ9L0+QYd/FfLxaTYUafLnpKlgLte3oN75N6zsV0DhvxAqJAf0Y/8CA97cqYyrrqz1/sJ58S7XxaTNjQkNaNzUd2AeeKJJ/TEE0+YXQZM0rR7vfy1FUo+a0G75Q2Fb8v9+aeKG3+e0i+/TRZLy7jLIxte0+E3H1HNe09p0Jzvdrl/R+ZIOTJHHrfc8Ht15NN/S1abEvJntntt8LyfdthRPrLpTR1e/kfVrinQ4Hk/bfda/PhzVf3OY6rf+Don0OixSTktJ5OtV/ei2ebSlvc4aYCdQI/LStKPZo3Rg2/07ukFP5o1RuOykkJUVWQKd35I0qBLvq+Y1CGyJ2eo9qPnVbPqxLM8+uoqdXDZb2VLGKTMa+6VPSmj3etGwH/cNuQHQoH8iH7kR//VFxPvom9F9RwwGNgaNq+QZFHc2OntltdvekOSlHref7SdPEtSwpRLZE/JUsOWlQp4PT0+buOOjxRoqlPsqGnHDTnsbDhf61VLb/WB416zxSXLlTtRjdvXKNDc1OO6MLDlD2k5mSw53KSaxmaTqwmf6oZmlVa3/J6cPMBOoCXp5vNGacHU3B5vv2Bqrr59/qgQVhSZ+iI/YodPkT05o+sVj6r9cKmM5kalzf7Occ0XSR0+4pT8QCiQHwMD+dE/9cXEu+hbNGAQlQzDkHtfoexpQ2VzJXyx3Ncsz4EdsqflHHfia7FY5Bo+RYbXreby4h4fu37zm5KkhG5MkNW0a60ktbv96FjOoeNaat+/tcd1YWBLjovRSWlxkqR1e6pNriZ81u1teW/DBsUpOTbG5Gr6ntVq0f1XTdTts8fKZrXIFsTMiq3r3T57rO6/amK7xsJAZGZ+nKimxm1rZI1NUuywyfKU71TdJ/9S7ccvqmnPRhlGoNNtyQ/0FvkxMJAf/U9fTbyLvkUbDVHJW1WigPuIYkee1n559QHJCCgmNbvD7Vrvq/RVl0m5+d0+rq+2Uu69m2RLTFfsl+Z/OVbD1vflrdonw+uR99A+NX3+qezJmUqecV2H6zuyWh5f5yndesL9Aicyc3yGHl+zR8+tLdFFEzLNLicslqzdJ0m6cFzwIwuijdVq0XcuGK2Z4zP08Js79NbWCgUMyW61yHd0gsXWn60Waea4DP3oYoaNtzIrP07EV1uhgPuIHFl5qlr+P6rftLzd647MURp81S9kTzp+glTyA6FAfgwM5Ef/0lcT76Jv0YBBVPIfOSRJsn7piRQBT8sjFK3O+A63szji2q3XXfWFKyQjoIT8mR0OB2/VuO19Ne74oO3Pjqw8pX/lDsWkZHW4fuuTNXxH3xfQE9efOUyPr9mjd7ZVqLS6UTmpcWaXFFIlhxv19tFJBK8/s+PRZAPJuKwk/e1rU3Wgtkmriw+paH+tyuvckqSsJJfyhyZrRl66spNjTa60fzErP04k0FAjSWqu2CXv4VINuvSHis07U4anQbUfLlX9pjd0cNlvlf21h47blvxAKJAfAwv5Yb6aD56T1DcT76Jv0YBBVAo0HZHU+YlyOBhGQPWFb0myKH7SrBOuO/jKn0mSAu56NVfsVs17/9CBJ3+owVf+TLHDJh+3vvXoMPhAU13I68bAMWpwgqaPHqQ1O6v07Cf7dPvscV1vFEGe/WSfDEOaMTpdowYndL3BAJGdHKv5U3M1vxf39g8kZuRHVwzj6KNhjYBSZlz/xaOqXQkaNOd7aq7co+YD2+Uu/UyunJPbbUt+IBTIj4GJ/DBHwOtW7ftPS2Li3WjEHDCISq2T3Rp+b7vlVmfrFcqGDrczmhvbrdcd7j0b5a87KNewSZ2OZPkyqytBrmGTlLHgHlntDlW9+rAMv+/4unwtk95ZYpzdrgs41g1Hr+wtWVsij+/4p6ZEKo/PryVrSyRx9RK9Y0Z+dOXYfcbmnXHc63GjT5ckNR/YeXxd5AdChPwA+gYT70Y3GjCISta4ltnrW69ktrKnZEkWa4dPG5Ikb3VZy3o9GOpXv+no5LuTLu72tlZnnBxDxspfXyVfB7UF3C3vwxY38GblR2hdND5TWUkuHapv1r83d/x7EIle3XRAVQ3Nyk526aLxA/f+ffSeGfnRlZjUbMnScsrW0cgcq6tlmeE7/glM5AdChfwAwo+Jd6MfDRhEJUf6SS0nyof3t1tujXHKkZ0n3+FS+Wor271mGIbcezbKEuNqm7QwWP6mOjXu/EhWV6Lixpzdo5r99YdbfrAdP3dM6/uISefKDHrHbrPqhrNa/h3d99o2VTdE/iNFqxua9dvXt0lquXpptxFt6Lm+zo9gWOwOOYeOlyR5q/Yd93rzoZar9/bk4ydHJT8QKuQHEH5MvBv9+JZBVLK6EhQzeLiay4uPezxn4uQ5kqTqVU9+cV+9pPqNr8tXU674CefLesxQbcPvk7eqpNOrnpLUUPSu5Pcp/uTzZbF3/OjCgKdR3qrSDl+r3/ymmg/skD11SIcTbXnKtkuSXCdN7LQGIFjfOGeE8jISdKjeo7tf+czscnrtly9/pkP1HuVlJOgb54wwuxxEuL7Oj2AlnnKpJKl2dYEM3xe3R3mrStRQ9JYsjli5vvTkJon8QGiRH0D4MPHuwMBNZYhacWPOUu3qZ+TZv12unPFty+MnzlTDtvfVuHWVymvL5cqdKF91mRp3fCh7cqZSzr2h3X789VUqe+xm2ZIylHPz/3V4rPrNR28/OsFQwYD7iMoeu1mO7NGKScuRLXFQyyS8B4rVXLFLFkec0ufectx2hmHIvXeTYgblKiZtaE8+CqAdp92mB+dP1lWPrNFLG8t06cRszT45uHmL+pvlReV6eVOZbFaLFs+fLKe986ePAcHqi/w4sukNeUq3SJK8B/dIarmV1b2vUJLkzJnQbvh53PhzFbfjAzVuX6Oyx7+n2BGnKOBpVOOOD2T4vBo09xbZXO0nDyU/EGrkBxAeTLw7cDACBpHnmKuOJ5Iw6WLJalPDlnfbLbdYrMq46hdKnr5IgcY61a1bJs/+rUqYNEtZNyzu9n3ynrLt8h7aK0f2GDkGD+90PWtsspLPvkYWu1PuPRtV98kyNXy2Uobfq8SpV2jIfz3SNsS83f5LiuSvO6iEKXOCL+roZ2SxWLr1XjBwTMlN0bfOGyVJ+vm/iiJyKPnhhmbduazlP6vfOnekJuemmFsQ+r9+lB+e0i1qKHpbDUVvq7liV8uy/V8sa23OfHFsi9K/codSL/yGLFarjmxcrsYdH8o5ZJwyr/mNEk6+4PhjkB8IA/IDA1KQ+dFTUTHxLvkRlAj928VA5HK5JKnDpwR1xJ44SPHjz1Xj1veUet6N7Z4iYbHHKGXGIqXMWNT1fpIzNezHr3b6unPI2BO+3srqcCnlnOuCqv1Y9ZvekNWVqIT8mUFv0/r0jtbPDOjID2bm6a0tFSqurNcdL2zWo9efJps1MkLTHzD04xc261B9s/IyEvSDi0I/7waiR3/Mj/S5t3Q46vFELFabkqbNU9K0eUGtT34gXMgPDBTdzY+eMAwjKibeJT+CwwgYRIy2L8AOnvLQmZRzbpDha9aRT7tukPRH3sP71bD1PSWfvVDWLw0tP5HWx47yBYgTccW0DCWPsVm0YkuFfvZiYbt5LforwzD0sxcLtWJLhRw2K0PH0SXyg/xAaJEfGCh6kh/dZbFYlLnofuX+cGnYjtEXyI/g0IBBxMjIaHk0oL+hOuht7MkZGnTpLbI4YsNVVlj5jhxS8vRrlXjq3G5t1/pEpdbPDOjMlNwU/fGaU2S1SEvWleieV7f065NowzB0z6tbtGRdiawW6Y/XTmHoOLpEfgSP/ECwyA8MBD3Jj55w5ea3G20ZiciP4NCAQcRISEjQ4Iws+Q6XdWu7+PHnKOm0y8NUVXjFDpuslOnXymLr+MlKnfFVlykjM1vx8fFhqgzR5JKJ2br/6kmSpMfX7NFPXiiUP9D/TqJbh40/vmaPJOmBqydpTn62uUUhIpAfwSM/0B3kB6JdT/NjICI/gkMDBhElL2+0vNV8AXbFW12mvLzRZpeBCLJgaq5+99VJbVcyb3p6fb+aWPFwQ7Nuenq9lq4rldUiPfjVSZo/NdfsshBByI/gkB/oLvID0Y78CA75ERwaMIgoY8fkSXXlZpfR/9WVt3xWQDcsmJqrPy86te2e/lm/X6XlReb/vi0vKtfFv1+lFVsqFGOz6JHrTuXkGd1GfgSJ/EAPkB+IZuRHkMiPoNCAQUTJz8+Xp3KPAs1NZpfSbwWam+Sp3Kv8/HyzS0EEumRitp6/6WyNzkjQofqWq4Y/eG6DKVczqxua9f1nN+imp9e3Pa3inzedzbBx9Aj50TXyA71BfiBakR9dIz+CRwMGEWXevHnyez1q2rXO7FL6raZda+X3ejRv3jyzS0GEmpKbole/N0M3nz9KVov00sYyzfr9e3phfak8Pn/Yj+/x+fXC+lLN+v17enlTmawW6dvnj9Ir35vBhInoMfKja+QHeov8QDQiP7pGfgTPYvTn6cqBDkw59TTtbIpV+hU/NbuUfunQst8qL96tDesJCfTeppIa3fb8JhVX1kuSBsU7tHBarhadcZJyUk88W39js08T7npDkrTlntmKc9hPuH5pdaMKPt6nJWtLVHX0imleRoIWz5/MiTNCgvw4MfIDoUR+IJqQHydGfgSPETCIONcuXCD37nUKeBrNLqXfCXga5f58na5duMDsUhAlJuem6JXvzdDts8cqK8mlqoZmPbJyl8793bv6xpNrtWJLhWoaez68vKaxWSu2VOgbT67VOb97V4+s3KWqhmZlJ7t0++yxevX7XLVE6JAfnSM/EGrkB6IJ+dE58qN7GAGDiFNaWqpRo/PkmnKZUs+/0exy+pXqlY/Ls+k17SzeoZycHLPLQZTx+QN6a2ulnv5or1bvPNTutZzUWE3KSVb+0GRNHJqs4YPiFTAMnffgSknSqtvPl9Vi0Z6qBhXur1XR/lptLq1VaXX7+6lnjE7X9WcO00XjM2S3cY0AoUV+dI78QDiRH4h05EfnyI/uoQGDiHTvvffql3ffrawb/0cx6cxmL0neQyUqf+K7+tXdd+vOO+80uxxEuV0H6/X0R3v1zrZK7a3q3dWgYYPidOG4DF1/5jCNGpwQogqBjpEfxyM/0JfID0Qq8uN45Ef30YBBRHK73Ro34WQdNBKVPv8eWaw2s0sylRHw69DzdynDWq+tnxXJ5XKZXRIGkNomrz7bX6vN+2vbrk5W1Lnl9gbareeKsSozyaX8ocmadPRK58lDk5UcG2NS5RiIyI/2yA+YifxAJCE/2iM/eoYGDCLWihUrNGfOHMVNOF9pl/xgwH4JGgG/Dr/+BzVuWanly5dr1qxZZpcESJICgYBq3V5JUrIrRlYrQ8LRP5AfLcgP9FfkB/or8qMF+dFzfJshYs2aNUsFBQVq3LJSh1//g4xA+B9v2N8c++VXUFDAlx/6FavVqtQ4p1LjnJw8o18hP8gP9G/kB/or8oP86K0TP9MN6OcWLlwoSVq0aJEC9VVKmXnTgLkn03uoRDVvPyr3vkIVFBS0fRYAgK6RH+QHAPQE+UF+9Aa3ICEqrFixQt/81k0q2bdXCVPnKfmshbI648wuKywCnkbVfrhE9euW6aRhw/W3R/9C5xkAeoj8ID8AoCfID/KjJ2jAIGq43W4tXrxY9/7mPvkCAblGTFXs2OmKHTVNVkes2eX1SqC5SU271qpp+xq5P18nu9WqX9z5c916661MeAUAvUR+AAB6gvxAd9GAQdQpLS1VQUGBnl2yVBs/XS9bjFPOjGFSUpZiUofInjpEtoQ0WewOWWwxksVidsktDEOG3yvD1yx//WH5qsvkrS6T6srlqdwrv9ejKaeepmsXLtCiRYuUk5NjdsUAEFXIDwBAT5AfCBYNGES13bt366WXXlJRUZG27yhWcfFOVVYcMLusoGRkZisvb7TGjslTfn6+rrjiCo0cOdLssgBgQCA/AAA9QX7gRGjAYMBpaGhQZWWl3G633G63+suvgMVikcvlksvlUkZGhuLj480uCQBwDPIDANAT5Ada0YABAAAAAAAIM6vZBQAAAAAAAEQ7GjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZjRgAAAAAAAAwowGDAAAAAAAQJjRgAEAAAAAAAgzGjAAAAAAAABhRgMGAAAAAAAgzGjAAAAAAAAAhBkNGAAAAAAAgDCjAQMAAAAAABBmNGAAAAAAAADCjAYMAAAAAABAmNGAAQAAAAAACDMaMAAAAAAAAGFGAwYAAAAAACDMaMAAAAAAAACEGQ0YAAAAAACAMKMBAwAAAAAAEGY0YAAAAAAAAMKMBgwAAAAAAECY0YABAAAAAAAIMxowAAAAAAAAYUYDBgAAAAAAIMxowAAAAAAAAIQZDRgAAAAAAIAwowEDAAAAAAAQZv8foTrvmYSlr3cAAAAASUVORK5CYII=\n" }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = qml.draw_mpl(circuit, decimals=2)(params)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have defined our variational circuit QNode, we can construct\n", "a function that computes the gradient of the $i\\text{th}$ parameter\n", "using the parameter-shift rule.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-0.06518877224958124\n" ] } ], "source": [ "def parameter_shift_term(qnode, params, i):\n", " shifted = params.copy()\n", " shifted[i] += np.pi/2\n", " forward = qnode(shifted) # forward evaluation\n", "\n", " shifted[i] -= np.pi\n", " backward = qnode(shifted) # backward evaluation\n", "\n", " return 0.5 * (forward - backward)\n", "\n", "# gradient with respect to the first parameter\n", "print(parameter_shift_term(circuit, params, 0))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to compute the gradient with respect to *all* parameters, we\n", "need to loop over the index `i`:\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[-6.51887722e-02 -2.72891905e-02 0.00000000e+00 -9.33934621e-02\n", " -7.61067572e-01 4.16333634e-17]\n" ] } ], "source": [ "def parameter_shift(qnode, params):\n", " gradients = np.zeros([len(params)])\n", "\n", " for i in range(len(params)):\n", " gradients[i] = parameter_shift_term(qnode, params, i)\n", "\n", " return gradients\n", "\n", "print(parameter_shift(circuit, params))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can compare this to PennyLane\\'s *built-in* quantum gradient support\n", "by using the `qml.grad `\n", "function, which allows us to compute gradients of hybrid\n", "quantum-classical cost functions. Remember, when we defined the QNode,\n", "we specified that we wanted it to be differentiable using the\n", "parameter-shift method (`diff_method=\"parameter-shift\"`).\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-0.06518877224958125\n" ] } ], "source": [ "grad_function = qml.grad(circuit)\n", "print(grad_function(params)[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Alternatively, we can directly compute quantum gradients of QNodes using\n", "PennyLane\\'s built in\n", "`qml.gradients `\n", "module:\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[-6.51887722e-02 -2.72891905e-02 -2.77555756e-17 -9.33934621e-02\n", " -7.61067572e-01 4.16333634e-17]\n" ] } ], "source": [ "print(qml.gradients.param_shift(circuit)(params))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you count the number of quantum evaluations, you will notice that we\n", "had to evaluate the circuit `2*len(params)` number of times in order to\n", "compute the quantum gradient with respect to all parameters. While\n", "reasonably fast for a small number of parameters, as the number of\n", "parameters in our quantum circuit grows, so does both\n", "\n", "1. the circuit depth (and thus the time taken to evaluate each\n", " expectation value or \\'forward\\' pass), and\n", "2. the number of parameter-shift evaluations required.\n", "\n", "Both of these factors increase the time taken to compute the gradient\n", "with respect to all parameters.\n", "\n", "### Benchmarking\n", "\n", "Let\\'s consider an example with a significantly larger number of\n", "parameters. We\\'ll make use of the\n", "`~pennylane.StronglyEntanglingLayers`\n", "template to make a more complicated QNode.\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "180\n", "0.8947771876917631\n" ] } ], "source": [ "dev = qml.device(\"default.qubit\", wires=4)\n", "\n", "@qml.qnode(dev, diff_method=\"parameter-shift\")\n", "def circuit(params):\n", " qml.StronglyEntanglingLayers(params, wires=[0, 1, 2, 3])\n", " return qml.expval(qml.PauliZ(0) @ qml.PauliZ(1) @ qml.PauliZ(2) @ qml.PauliZ(3))\n", "\n", "# initialize circuit parameters\n", "param_shape = qml.StronglyEntanglingLayers.shape(n_wires=4, n_layers=15)\n", "params = np.random.normal(scale=0.1, size=param_shape, requires_grad=True)\n", "print(params.size)\n", "print(circuit(params))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This circuit has 180 parameters. Let\\'s see how long it takes to perform\n", "a forward pass of the circuit.\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Forward pass (best of 3): 0.01652761589939473 sec per loop\n" ] } ], "source": [ "import timeit\n", "\n", "reps = 3\n", "num = 10\n", "times = timeit.repeat(\"circuit(params)\", globals=globals(), number=num, repeat=reps)\n", "forward_time = min(times) / num\n", "\n", "print(f\"Forward pass (best of {reps}): {forward_time} sec per loop\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now estimate the time taken to compute the full gradient vector,\n", "and see how this compares.\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Gradient computation (best of 3): 6.592562994100445 sec per loop\n" ] } ], "source": [ "# create the gradient function\n", "grad_fn = qml.grad(circuit)\n", "\n", "times = timeit.repeat(\"grad_fn(params)\", globals=globals(), number=num, repeat=reps)\n", "backward_time = min(times) / num\n", "\n", "print(f\"Gradient computation (best of {reps}): {backward_time} sec per loop\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Based on the parameter-shift rule, we expect that the amount of time to compute the quantum gradients should be approximately $2p\\Delta t_{f}$ where $p$ is the number of parameters and $\\Delta t_{f}$ if the time\n", "taken for the forward pass. Let\\'s verify this:\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "5.9499417237821035\n" ] } ], "source": [ "print(2 * forward_time * params.size)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Backpropagation example\n", "\n", "\n", "An alternative to the parameter-shift rule for computing gradients is [reverse-mode\n", "autodifferentiation](https://en.wikipedia.org/wiki/Reverse_accumulation). Unlike the parameter-shift method, which requires $2p$ circuit evaluations for $p$ parameters, reverse-mode requires only a *single* forward pass of the differentiable function to compute the gradient of all variables, at the expense of increased memory usage. During the forward pass, the results of all intermediate subexpressions are stored; the computation is then traversed *in reverse*, with the gradient computed by repeatedly applying the chain rule. In most classical machine learning settings (where we are training scalar loss functions consisting of a large number of parameters), reverse-mode\n", "autodifferentiation is the preferred method of autodifferentiation\\-\\--the reduction in computational time enables larger and more complex models to be successfully trained. The\n", "backpropagation algorithm is a particular special-case of reverse-mode autodifferentiation, which has helped lead to the machine learning explosion we see today.\n", "\n", "In quantum machine learning, however, the inability to store and utilize the results of *intermediate* quantum operations on hardware remains a barrier to using backprop; while reverse-mode autodifferentiation works\n", "fine for small quantum simulations, only the parameter-shift rule can be used to compute gradients on quantum hardware directly. Nevertheless, when training quantum models via classical simulation, it\\'s useful to\n", "explore the regimes where reverse-mode differentiation may be a better choice than the parameter-shift rule.\n", "\n", "### Benchmarking\n", "\n", "\n", "When creating a QNode, `PennyLane supports various methods of differentiation\n", "`, including `\"parameter-shift\"` (which we used previously), `\"finite-diff\"`, `\"reversible\"`, and `\"backprop\"`. While `\"parameter-shift\"` works with all devices (simulator or hardware), `\"backprop\"` will only work for specific simulator devices that are designed to support backpropagation.\n", "\n", "One such device is `default.qubit `. It has backends written using TensorFlow, JAX, and\n", "Autograd, so when used with the TensorFlow, JAX, and Autograd interfaces respectively, supports backpropagation. In this demo, we will use the default Autograd interface.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "dev = qml.device(\"default.qubit\", wires=4)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When defining the QNode, we specify `diff_method=\"backprop\"` to ensure that we are using backpropagation mode. Note that this is the *default differentiation mode* for the `default.qubit` device.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "@qml.qnode(dev, diff_method=\"backprop\")\n", "def circuit(params):\n", " qml.StronglyEntanglingLayers(params, wires=[0, 1, 2, 3])\n", " return qml.expval(qml.PauliZ(0) @ qml.PauliZ(1) @ qml.PauliZ(2) @ qml.PauliZ(3))\n", "\n", "# initialize circuit parameters\n", "param_shape = qml.StronglyEntanglingLayers.shape(n_wires=4, n_layers=15)\n", "params = np.random.normal(scale=0.1, size=param_shape, requires_grad=True)\n", "print(circuit(params))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let\\'s see how long it takes to perform a forward pass of the circuit.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import timeit\n", "\n", "reps = 3\n", "num = 10\n", "times = timeit.repeat(\"circuit(params)\", globals=globals(), number=num, repeat=reps)\n", "forward_time = min(times) / num\n", "print(f\"Forward pass (best of {reps}): {forward_time} sec per loop\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Comparing this to the forward pass from `default.qubit`, we note that there is some potential overhead from using backpropagation. We can now time how long it takes to perform a gradient computation via\n", "backpropagation:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "times = timeit.repeat(\"qml.grad(circuit)(params)\", globals=globals(), number=num, repeat=reps)\n", "backward_time = min(times) / num\n", "print(f\"Backward pass (best of {reps}): {backward_time} sec per loop\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Unlike with the parameter-shift rule, the time taken to perform the backwards pass appears of the order of a single forward pass! The can significantly speed up training of simulated circuits with many\n", "parameters.\n", "\n", "## Send it after class: Time comparison\n", "\n", "\n", "Let\\'s compare the two differentiation approaches as the number of trainable parameters in the variational circuit increases, by timing both the forward pass and the gradient computation as the number of\n", "layers is allowed to increase.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "dev = qml.device(\"default.qubit\", wires=4)\n", "\n", "def circuit(params):\n", " qml.StronglyEntanglingLayers(params, wires=[0, 1, 2, 3])\n", " return qml.expval(qml.PauliZ(0) @ qml.PauliZ(1) @ qml.PauliZ(2) @ qml.PauliZ(3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We\\'ll continue to use the same ansatz as before, but to reduce the time taken to collect the data, we\\'ll reduce the number and repetitions of timings per data point. Below, we loop over a variational circuit depth ranging from 0 (no gates/ trainable parameters) to 20. Each layer will contain $3N$ parameters, where $N$ is the number of wires (in this case, we have $N=4$).\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "reps = #fill me\n", "num = #fill me\n", "\n", "forward_shift = []\n", "gradient_shift = []\n", "forward_backprop = []\n", "gradient_backprop = []\n", "\n", "for depth in range(0, -fill me-):\n", " param_shape = qml.StronglyEntanglingLayers.shape(n_wires=4, n_layers=depth)\n", " params = np.random.normal(scale=0.1, size=param_shape, requires_grad=True)\n", " num_params = params.size\n", "\n", " # forward pass timing\n", " # ===================\n", "\n", " qnode_shift = #fill me\n", " qnode_backprop = #fill me\n", "\n", " # parameter-shift\n", " t = timeit.repeat(\"qnode_shift(params)\", globals=globals(), number=num, repeat=reps)\n", " forward_shift.append([num_params, min(t) / num])\n", "\n", " # backprop\n", " t = timeit.repeat(\"qnode_backprop(params)\", globals=globals(), number=num, repeat=reps)\n", " forward_backprop.append([num_params, min(t) / num])\n", "\n", " if num_params == 0:\n", " continue\n", "\n", " # Gradient timing\n", " # ===============\n", "\n", " qnode_shift = #fill me\n", " qnode_backprop = #fill me\n", "\n", " # parameter-shift\n", " t = timeit.repeat(\"qml.grad(qnode_shift)(params)\", globals=globals(), number=num, repeat=reps)\n", " gradient_shift.append([num_params, min(t) / num])\n", "\n", " # backprop\n", " t = timeit.repeat(\"qml.grad(qnode_backprop)(params)\", globals=globals(), number=num, repeat=reps)\n", " gradient_backprop.append([num_params, min(t) / num])\n", "\n", "gradient_shift = np.array(gradient_shift).T\n", "gradient_backprop = np.array(gradient_backprop).T\n", "forward_shift = np.array(forward_shift).T\n", "forward_backprop = np.array(forward_backprop).T" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now import matplotlib, and plot the results.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "plt.style.use(\"bmh\")\n", "\n", "fig, ax = plt.subplots(1, 1, figsize=(6, 4))\n", "\n", "ax.plot(*gradient_shift, '.-', label=\"Parameter-shift\")\n", "ax.plot(*gradient_backprop, '.-', label=\"Backprop\")\n", "ax.set_ylabel(\"Time (s)\")\n", "ax.set_xlabel(\"Number of parameters\")\n", "ax.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that the computational time for the parameter-shift rule increases with increasing number of parameters, as expected, whereas the computational time for backpropagation appears much more constant, with\n", "perhaps a minute linear increase with $p$. Note that the plots are not perfectly linear, with some \\'bumpiness\\' or noisiness. This is likely due to low-level operating system jitter, and other environmental\n", "fluctuations\\-\\--increasing the number of repeats can help smooth out the plot.\n", "\n", "For a better comparison, we can scale the time required for computing the quantum gradients against the time taken for the corresponding forward pass:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "gradient_shift[1] /= forward_shift[1, 1:]\n", "gradient_backprop[1] /= forward_backprop[1, 1:]\n", "\n", "fig, ax = plt.subplots(1, 1, figsize=(6, 4))\n", "\n", "ax.plot(*gradient_shift, '.-', label=\"Parameter-shift\")\n", "ax.plot(*gradient_backprop, '.-', label=\"Backprop\")\n", "\n", "# perform a least squares regression to determine the linear best fit/gradient\n", "# for the normalized time vs. number of parameters\n", "x = gradient_shift[0]\n", "m_shift, c_shift = np.polyfit(*gradient_shift, deg=1)\n", "m_back, c_back = np.polyfit(*gradient_backprop, deg=1)\n", "\n", "ax.plot(x, m_shift * x + c_shift, '--', label=f\"{m_shift:.2f}p{c_shift:+.2f}\")\n", "ax.plot(x, m_back * x + c_back, '--', label=f\"{m_back:.2f}p{c_back:+.2f}\")\n", "\n", "ax.set_ylabel(\"Normalized time\")\n", "ax.set_xlabel(\"Number of parameters\")\n", "ax.set_xscale(\"log\")\n", "ax.set_yscale(\"log\")\n", "ax.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now see clearly that there is constant overhead for backpropagation with `default.qubit`, but the parameter-shift rule scales as $\\sim 2p$." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" } }, "nbformat": 4, "nbformat_minor": 0 }