{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# [cknowledge.org](http://cknowledge.org): Community-driven benchmarking and optimization of computing systems: from classical to quantum"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Quantum Computing](https://github.com/ctuning/ck-quantum/wiki)\n",
"* [CK-QISKit](https://github.com/ctuning/ck-qiskit) (IBM)\n",
"* [CK-Rigetti](https://github.com/ctuning/ck-rigetti)\n",
"* [CK-ProjectQ](https://github.com/ctuning/ck-projectq)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Artificial Intelligence and Machine Learning](http://cknowledge.org/ai)\n",
"* [Reproducible Quality-Efficient Systems Tournaments](http://cknowledge.org/request) ([ReQuEST initiative](http://cknowledge.org/request.html#organizers))\n",
"* [AI artifacts](http://cknowledge.org/ai-artifacts) (cTuning foundation)\n",
"* [Android app](https://play.google.com/store/apps/details?id=openscience.crowdsource.video.experiments) (dividiti)\n",
"* [Desktop app](https://github.com/dividiti/ck-crowdsource-dnn-optimization) (dividiti)\n",
"* [CK-Caffe](https://github.com/dividiti/ck-caffe) (Berkeley)\n",
"* [CK-Caffe2](https://github.com/ctuning/ck-caffe2) (Facebook)\n",
"* [CK-CNTK](https://github.com/ctuning/ck-cntk) (Microsoft)\n",
"* [CK-KaNN](https://github.com/ctuning/ck-kann) (Kalray)\n",
"* [CK-MVNC](https://github.com/ctuning/ck-mvnc) (Movidius / Intel)\n",
"* [CK-MXNet](https://github.com/ctuning/ck-mxnet) (Apache)\n",
"* [CK-NNTest](https://github.com/ctuning/ck-nntest) (cTuning foundation)\n",
"* [CK-TensorFlow](https://github.com/ctuning/ck-tensorflow) (Google)\n",
"* [CK-TensorRT](https://github.com/ctuning/ck-tensorrt) (NVIDIA)\n",
"* etc."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# VQE demo on Rigetti QVM"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Table of Contents"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. [Overview](#overview)\n",
"1. [Experimental platform](#platform)\n",
"1. [Experimental data](#data) [for developers]\n",
"1. [Data wrangling code](#code) [for developers]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Overview"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This Jupyter Notebook studies the performance of [Variational-Quantum-Eigensolver](http://grove-docs.readthedocs.io/en/latest/vqe.html) (VQE) on Rigetti's [Quantum Virtual Machine](http://pyquil.readthedocs.io/en/latest/qvm.html) (QVM)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Platform"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"_TODO_"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Get the experimental data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The experimental data can be downloaded and registered with CK as described below."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"```\n",
"$ wget https://www.dropbox.com/s/<...>/ck-rigetti-qvm-vqe.zip\n",
"$ ck add repo --zip=ck-rigetti-qvm-vqe.zip\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"repo_uoa = 'ck-rigetti-qvm-vqe'\n",
"!ck list $repo_uoa:experiment:* --print_full | sort"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Data wrangling code"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**NB:** Please ignore this section if you are not interested in re-running or modifying this notebook."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Includes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Standard"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
"import json\n",
"import re"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Scientific"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If some of the scientific packages are missing, please install them using:\n",
"```\n",
"# pip install jupyter pandas numpy matplotlib\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import IPython as ip\n",
"import pandas as pd\n",
"import numpy as np\n",
"import matplotlib as mp\n",
"import seaborn as sb"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print ('IPython version: %s' % ip.__version__)\n",
"print ('Pandas version: %s' % pd.__version__)\n",
"print ('NumPy version: %s' % np.__version__)\n",
"print ('Matplotlib version: %s' % mp.__version__)\n",
"print ('Seaborn version: %s' % sb.__version__)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import Image, display\n",
"def display_in_full(df):\n",
" pd.options.display.max_columns = len(df.columns)\n",
" pd.options.display.max_rows = len(df.index)\n",
" display(df)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"default_colormap = cm.autumn\n",
"default_fontsize = 16\n",
"default_barwidth = 0.8\n",
"default_figwidth = 16\n",
"default_figheight = 8\n",
"default_figdpi = 200\n",
"default_figsize = [default_figwidth, default_figheight]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if mp.__version__[0]=='2': mp.style.use('classic')\n",
"mp.rcParams['figure.max_open_warning'] = 200\n",
"mp.rcParams['figure.dpi'] = default_figdpi\n",
"mp.rcParams['font.size'] = default_fontsize\n",
"mp.rcParams['legend.fontsize'] = 'medium'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Collective Knowledge"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If CK is not installed, please install it using:\n",
"```\n",
"# pip install ck\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import ck.kernel as ck\n",
"print ('CK version: %s' % ck.__version__)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Access experimental data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def get_experimental_results(repo_uoa, tags='explore-method-iteration-sample', module_uoa='experiment'):\n",
" r = ck.access({'action':'search', 'repo_uoa':repo_uoa, 'module_uoa':module_uoa, 'tags':tags})\n",
" if r['return']>0:\n",
" print('Error: %s' % r['error'])\n",
" exit(1)\n",
" experiments = r['lst']\n",
" \n",
" dfs = []\n",
" for experiment in experiments:\n",
" skip = False\n",
" data_uoa = experiment['data_uoa']\n",
" r = ck.access({'action':'list_points', 'repo_uoa':repo_uoa, 'module_uoa':module_uoa, 'data_uoa':data_uoa})\n",
" if r['return']>0:\n",
" print('Error: %s' % r['error'])\n",
" exit(1)\n",
" tags = r['dict']['tags']\n",
" \n",
" platform_tags = [ tag for tag in tags if tag in ['rigetti-qvm'] ]\n",
" if len(platform_tags)!=1: skip = True\n",
" \n",
" program_tags = [ tag for tag in tags if tag in ['vqe-helium-example'] ]\n",
" if len(program_tags)!=1: skip = True\n",
" \n",
" if skip:\n",
" print('[Warning] Bad library tags. Skipping experiment with tags:')\n",
" print(tags)\n",
" continue\n",
" # For each point. \n",
" for point in r['points']:\n",
" point_file_path = os.path.join(r['path'], 'ckp-%s.0001.json' % point)\n",
" with open(point_file_path) as point_file:\n",
" point_data_raw = json.load(point_file)\n",
" characteristics_list = point_data_raw['characteristics_list']\n",
" num_repetitions = len(characteristics_list)\n",
" data = [\n",
" {\n",
" # features\n",
" 'platform': platform_tags[0],\n",
" # choices\n",
" 'minimizer_method': characteristics['run'].get('vqe_input', {}).get('minimizer_method', 'n/a'),\n",
" 'minimizer_options': characteristics['run'].get('vqe_input', {}).get('minimizer_options', {'maxfev':-1}),\n",
" 'sample_number': characteristics['run'].get('vqe_input', {}).get('sample_number','n/a'),\n",
" # statistical repetition\n",
" 'repetition_id': repetition_id,\n",
" # runtime characteristics\n",
" 'report': characteristics['run'].get('report', {}),\n",
" 'vqe_output': characteristics['run'].get('vqe_output', {}),\n",
" }\n",
" for (repetition_id, characteristics) in zip(range(num_repetitions), characteristics_list)\n",
" if len(characteristics['run']) > 0\n",
" ]\n",
" for datum in data:\n",
" datum['success'] = datum.get('vqe_output',{}).get('success',False)\n",
" datum['nfev'] = np.int64(datum.get('vqe_output',{}).get('nfev',-1))\n",
" datum['nit'] = np.int64(datum.get('vqe_output',{}).get('nit',-1))\n",
" datum['fun'] = np.float64(datum.get('vqe_output',{}).get('fun',0))\n",
" datum['total_seconds'] = np.float64(datum.get('report',{}).get('total_seconds',0))\n",
" datum['total_q_seconds'] = np.float64(datum.get('report',{}).get('total_q_seconds',0))\n",
" datum['max_iterations'] = list(datum.get('minimizer_options',{'maxfev':-1}).values())[0]\n",
" index = [\n",
" 'minimizer_method', 'max_iterations', 'sample_number', 'repetition_id'\n",
" ]\n",
" # Construct a DataFrame.\n",
" df = pd.DataFrame(data)\n",
" df = df.set_index(index)\n",
" # Append to the list of similarly constructed DataFrames.\n",
" dfs.append(df)\n",
" if dfs:\n",
" # Concatenate all thus constructed DataFrames (i.e. stack on top of each other).\n",
" result = pd.concat(dfs)\n",
" result.sort_index(ascending=True, inplace=True)\n",
" else:\n",
" # Construct a dummy DataFrame the success status of which can be safely checked.\n",
" result = pd.DataFrame(columns=['success'])\n",
" return result"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plot experimental data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def plot(df, sample_number_set=None,\n",
" xmin=0.0, xmax=85.01, xstep=5.00,\n",
" ymin=-3.0, ymax=-0.49, ystep=0.25,\n",
" figsize=(18,9), dpi=200, legend_loc='center right'):\n",
" fig = plt.figure(figsize=figsize, dpi=dpi)\n",
" ax = fig.gca()\n",
"\n",
" # Options.\n",
" markersize = 10\n",
" sample_number_to_marker = {\n",
" 200 : 'o',\n",
" 500 : '^',\n",
" 1000 : 'd',\n",
" 2000 : 'p'\n",
" }\n",
" minimizer_method_to_color = {\n",
" 'COBYLA' : 'red',\n",
" 'Nelder-Mead' : 'green'\n",
" }\n",
" # repetition_id_to_linestyle = {\n",
" # 0 : '-',\n",
" # 1 : '--',\n",
" # 2 : ':',\n",
" # }\n",
" last_marker_size = 16\n",
" last_marker_color = 'black'\n",
" last_marker_success_true = '^'\n",
" last_marker_success_false = 'v'\n",
" if not sample_number_set: sample_number_set = sample_number_to_marker.keys()\n",
"\n",
" for index, row in df.iterrows():\n",
" (minimizer_method, max_iterations, sample_number, repetition_id) = index\n",
" if sample_number not in sample_number_set: continue\n",
" energies = [ iteration['energy'] for iteration in row['report']['iterations'] ]\n",
" marker=sample_number_to_marker[sample_number]\n",
" color=minimizer_method_to_color[minimizer_method]\n",
" markerfacecolor=color\n",
" linestyle='-' #repetition_id_to_linestyle[repetition_id]\n",
" markersize=markersize\n",
" ax.plot(range(len(energies)), energies, marker=marker, color=color, linestyle=linestyle,\n",
" markerfacecolor=markerfacecolor, markersize=markersize)\n",
" # Mark last function evaluation.\n",
" last_marker = last_marker_success_true if row['success'] else last_marker_success_false\n",
" ax.plot(row['nfev']-1, energies[-1], color=minimizer_method_to_color[minimizer_method],\n",
" marker=last_marker, markersize=last_marker_size)\n",
"\n",
" # Vertical lines for max_iterations.\n",
" for max_iterations in [60, 80]:\n",
" plt.axvline(x=max_iterations, color='black')\n",
" # Grid.\n",
" plt.grid()\n",
" # Title.\n",
" title = 'VQE on QVM'\n",
" ax.set_title(title)\n",
" # X axis.\n",
" xlabel='Function evaluation'\n",
" ax.set_xlabel(xlabel)\n",
" ax.set_xlim(xmin, xmax)\n",
" ax.set_xticks(np.arange(xmin, xmax, xstep))\n",
" # Y axis.\n",
" ylabel='Energy'\n",
" ax.set_ylabel(ylabel)\n",
" ax.set_ylim(ymin, ymax)\n",
" ax.set_yticks(np.arange(ymin, ymax, ystep))\n",
" # Legend. https://matplotlib.org/users/legend_guide.html\n",
" handles = [\n",
" mp.lines.Line2D([], [], color=minimizer_method_to_color[m],\n",
" marker=sample_number_to_marker[s], markersize=markersize,\n",
" label='minimizer_method=\"%s\",sample_number=%d' % (m,s))\n",
" for m in sorted(minimizer_method_to_color.keys())\n",
" for s in sorted(sample_number_to_marker.keys())\n",
" ]\n",
" plt.legend(handles=handles, title='minimizer_method,sample_number', loc=legend_loc)\n",
" # Save figure.\n",
" plt.savefig('vqe.energy.png')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def plot_time(df):\n",
" # \"df.index.names[:-1]\" means reduce along 'repetition_id' (statistical variation).\n",
" df_mean = df[['total_q_seconds','total_seconds']].groupby(level=df.index.names[:-1]).mean()\n",
" df_std = df[['total_q_seconds','total_seconds']].groupby(level=df.index.names[:-1]).std()\n",
" ax = df_mean.plot(kind='bar', yerr=df_std, grid=True, legend=True, rot=45,\n",
" fontsize=default_fontsize, figsize=default_figsize, colormap=default_colormap)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Analysis"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df = get_experimental_results(repo_uoa=repo_uoa)\n",
"display_in_full(df)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plot convergence"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, sample_number_set=[500])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, sample_number_set=[1000])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, sample_number_set=[2000])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, xmin=5, xmax=25.01, xstep=1, ymax=-2.59, ystep=0.05, legend_loc='lower left')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, xmin=5, xmax=25.01, xstep=1, ymax=-2.59, ystep=0.05, legend_loc='lower left',\n",
" sample_number_set=[200])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, xmin=5, xmax=25.01, xstep=1, ymax=-2.59, ystep=0.05, legend_loc='lower left',\n",
" sample_number_set=[500])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot(df, xmin=5, xmax=25.01, xstep=1, ymax=-2.59, ystep=0.05, legend_loc='lower left',\n",
" sample_number_set=[1000])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plot execution time"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot_time(df)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 1
}