# VeriQR **Repository Path**: herunhong/VeriQR ## Basic Information - **Project Name**: VeriQR - **Description**: No description available - **Primary Language**: Unknown - **License**: GPL-3.0 - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2024-05-24 - **Last Updated**: 2024-08-21 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README VeriQR === A Qt application for robustness verification of quantum classifiers, implemented in c++. It provides a user-friendly GUI, visualizing both inputs and outputs and providing an intuitive way to verify the robustness of quantum machine learning algorithms with respect to a small disturbance of noises, derived from the surrounding environment. Clone or download the tool [VeriQR](https://github.com/sekauqnoom/RobustnessVerifier.git): ```bash git clone https://github.com/Veri-Q/VeriQR.git ``` ## Requirements *You can compile VeriQR on Unix and Linux. The following installation instruction is based on Ubuntu 22.04* ### ubuntu dependency for qt6 ```bash # install cmake make gcc g++ clang llvm sudo apt update -y sudo apt-get install gcc g++ # install qt6-related dependencies sudo apt install -y build-essential libgl1-mesa-dev gdb sudo apt install -y pkg-config libssl-dev zlib1g-dev sudo apt install -y libxkbcommon-dev sudo apt install -y libvulkan-dev sudo apt install -y wget vim bash curl git # install qt6 sudo add-apt-repository universe sudo apt install -y qt6* sudo apt install -y libqt6* ``` ### qt6 qmake setting 1. Select Qt6 system-wide ```bash 1) vim ~/qt6.conf # Add the following information and save: qtchooser -install qt6 $(which qmake6) 2) sudo mv ~/qt6.conf /usr/share/qtchooser/qt6.conf ``` 2. Set Qt6 as default option ```bash sudo mkdir -p /usr/lib/$(uname -p)-linux-gnu/qt-default/qtchooser sudo rm /usr/lib/$(uname -p)-linux-gnu/qt-default/qtchooser/default.conf sudo ln -n /usr/share/qtchooser/qt6.conf /usr/lib/$(uname -p)-linux-gnu/qt-default/qtchooser/default.conf sudo rm /usr/bin/qmake sudo ln -s /usr/bin/qmake6 /usr/bin/qmake ``` 3. Select Qt6 as default (place in ~/.bashrc for persistence): ```bash 1) vim ~/.bashrc # Add the following information and save: export QT_SELECT=qt6 2) source ~/.bashrc ``` ### dependency for python module 1. Follow the instructions of [Miniconda Installation](https://conda.io/projects/conda/en/latest/user-guide/install/index.html) to install Miniconda. 2. Execute the following commands: ```bash sudo apt install -y libblas-dev liblapack-dev cmake python3-pip cd VeriQR/py_module conda create -n VeriQR python=3.9.0 conda activate VeriQR chmod +x install.sh ./install.sh ``` ###### Notice: In order to use the mindquantum.io.qasm module, we need to fix some bugs in source codes. 1. Replace the code in the `extra_params` method in the `/yourPathTo/miniconda3/envs/VeriQR/lib/python3.9/site-packages/mindquantum/io/qasm/openqasm.py` file ```python def extra_params(cmd, dtype=float): """Get gate parameters.""" matches = re.findall(r'\((.*)\)', cmd) out = [] for i in matches: for j in i.split(','): pr = j.strip() if dtype == str: out.append(pr) else: if '*' in pr: pr = pr.replace('pi', str(np.pi)).replace('π', str(np.pi)) pr = [float(i.strip()) for i in pr.split('*')] out.append(pr[0] * pr[1]) elif '/' in pr: pr = pr.replace('pi', str(np.pi)).replace('π', str(np.pi)) pr = [float(i.strip()) for i in pr.split('/')] out.append(pr[0] / pr[1]) else: out.append(float(pr)) return out ``` with the following code: ```python def extra_params(cmd, dtype=float): """Get gate parameters.""" matches = re.findall(r'\((.*)\)', cmd) out = [] for i in matches: for j in i.split(','): pr = j.strip() if dtype == str: out.append(pr) else: flag_1 = '*' in pr flag_2 = '/' in pr if flag_1: pr = pr.replace('pi', str(np.pi)).replace('π', str(np.pi)) pr = [i.strip() for i in pr.split('*')] if flag_2: result = 1 if not flag_1: pr = pr.replace('pi', str(np.pi)).replace('π', str(np.pi)) pr = [k.strip() for k in pr.split('/')] r = float(pr[0]) for k in range(len(pr) - 1): r = r / float(pr[k + 1]) result *= r else: for item in pr: items = [k.strip() for k in item.split('/')] r = float(items[0]) for k in range(len(items) - 1): r = r / float(items[k + 1]) result *= r out.append(result) else: if dtype == float: out.append(pr) else: out.append(float(pr)) return out ``` 2. Replace the code in `__init__` method in `class Power(NoneParamNonHermMat)` in the `/yourPathTo/miniconda3/envs/VeriQR/lib/python3.9/site-packages/mindquantum/core/gates/basicgate.py` file: ```python name = f'{gate}^{exponent}' ``` with the following code: ```python name = f'{gate}' + '^' + str(exponent) ``` ## Compile VeriQR ```bash cd VeriQR mkdir build cd build # If the 'VeriQR' env has been deactivated, you need to activate it again now: conda activate VeriQR qmake .. make -j8 ./VeriQR ``` ## User Manual See [veriqr_manual.pdf](https://github.com/Veri-Q/VeriQR/blob/main/veriqr_manual.pdf) # Experimental Results ## Local Robustness Verification VeriQR provides several labeled datasets, all of which are encoded into quantum data by different quantum encoders and trained on Mindspore or Tensorflow Quantum platforms to generate QML models for verification. In total, we verify the local-robustness of 50 quantum classifiers, including: - The model $qubit$, used to identify the region of a quantum state in the X-Z plane of the Bloch sphere. - The model $iris$, trained on the [Iris dataset](https://archive.ics.uci.edu/dataset/53/iris) that is encoded into mixed states of 4 qubits via Instantaneous Quantum Polynomial (IQP) encoding, is used for classifying irises of different subgenera. - The models $mnist$, trained on the well-known MNIST dataset that is encoded into pure states of 8 qubits using amplitude encoding, is used for classifying handwritten digits, which can be any two of numbers ranging from 0 to 9. - The model $fashion$, trained on the well-known Fashion MNIST dataset that is encoded into pure states of 8 qubits via amplitude encoding, is used to classify images of two fashion products, T-shirts and ankle boots. - The models $tfi$, trained on the data points of 4 qubits and of 8 qubits in the [TFI\_chain dataset](https://tensorflow.google.cn/quantum/api_docs/python/tfq/datasets/tfi_chain) respectively using the approach in [the paper](https://www.scipost.org/SciPostPhysLectNotes.61), are used for identifying the wavefunction at different phases in a quantum many-body system. We conducted numerous experiments on different circuits for each model as outlined in the [local_results](https://github.com/Veri-Q/VeriQR/blob/main/py_module/Local/results/local_results.csv) table: - The noiseless ideal QML model with quantum circuit $c_0$; - Circuit $c_1$ created by introducing random noise at various random points in circuit $c_0$ to simulate noise effects on NISQ devices; - Circuit $c_2$ modified by adding specific noise with a noise level $0 \leq p \leq 1$ of four types: *depolarizing*, *phase flip*, *bit flip*, and *mixed* (a combination of the three) noise (referred to as "noise_p" below $c_2$), applied to each qubit after the random noise manipulation on circuit $c_1$. Where $RA$ indicates the robust accuracy of classifiers, and $VT$ is the verification time in seconds. Here are some of the experimental results:
Model #Qubits ε Circuit Noise Setting (noise_p) Rough Verif Accurate Verif
RA (%) VT (sec.) RA (%) VT (sec.)
qubit 1 0.001 c_0 noiseless 88.12 0.0038 90 2.4226
c_1 random 88.12 0.0039 90 2.4623
c_2 depolarizing_0.001 88.00 0.0038 90 2.4873
c_2 depolarizing_0.005 87.62 0.0053 90 2.7140
iris 4 0.005 c_0 noiseless 98.75 0.0013 100 0.4924
c_1 random 97.50 0.0009 100 0.8876
c_2 mixed_0.01 97.50 0.0019 100 0.8808
c_2 mixed_0.05 96.25 0.0021 100 3.1675
tfi 4 0.005 c_0 noiseless 86.41 0.0039 100 6.5220
c_1 random 85.94 0.0038 100 6.6438
c_2 mixed_0.01 85.78 0.0061 100 6.7117
c_2 mixed_0.05 85.16 0.0063 100 7.0374
tfi 8 0.005 c_0 noiseless 98.44 0.0372 100 2.3004
c_1 random 96.56 0.1061 100 3.9492
c_2 bit-flip_0.01 96.56 37.0965 100 42.1246
c_2 bit-flip_0.05 95.94 32.7195 100 38.8139
fashion 8 0.001 c_0 noiseless 90.60 0.0420 97.40 25.3777
c_1 random 90.30 0.0934 97.30 27.4964
c_2 bit-flip_0.01 89.90 15.6579 97.20 42.1063
c_2 bit-flip_0.05 87.60 14.0342 96.70 48.5805
mnist (1&3) 8 0.003 c_0 noiseless 93.80 0.0543 96.00 18.5063
c_1 random 92.60 0.0785 95.70 23.2905
c_2 phase-flip_0.001 92.60 12.9728 95.70 36.2348
c_2 phase-flip_0.01 92.60 11.6704 95.70 33.7894
### Adversarial Training for Improving Robustness *VeriQR* empowers users with adversarial training capabilities, an extension of traditional machine learning. When the $\epsilon$-local robustness of $\rho$ with label $l$ is compromised, our robustness verification algorithms embedded in *VeriQR* automatically generate an adversarial example $\sigma$. By incorporating a number of states and their ground truth labels $(\sigma, l)$ into the training dataset, we retrained several QML model to enhance their local robustness against the adversarial examples. The results are shown as followed, demonstrating the effectiveness of adversarial training in boosting the local robustness of QMLs, where $NRN$ represents the number of non-robust states in the original dataset for each model. In particular, since most of our experimental models achieved 100% RA, we use "`-`" entries here to indicate that these models do not need adversarial training after initial validation.
Model #Qubits Circuit Noise   Setting
     (noise_p)		 ε Traning Rough Verif Accurate Verif
NRN RA (%) VT (sec.) NRN RA (%) VT (sec.)
fashion 8 c_0 noiseless 0.001 before 94 90.60 0.0471 26 97.40 19.9476
after 90 90.06 0.0450 15 98.54 27.0747
0.005 before 187 81.30 0.0207 44 95.60 38.1464
after 56 93.10 0.0331 27 97.41 17.0312
c_1 random 0.001 before 111 88.90 1.5387 30 97.00 24.2540
after 33 95.92 0.0310 10 99.03 10.5288
0.005 before 215 78.50 1.5210 50 95.00 54.8998
after 70 91.62 0.0329 38 96.38 21.5003
c_2 bit-flip_0.01 0.001 before 120 88.00 19.0693 24 97.60 44.5314
after 143 85.35 0.0380 13 98.73 41.7078
0.005 before 263 73.70 25.1982 41 95.90 81.4979
after 88 90.01 0.0316 34 96.73 26.1886
c_2 depolarizing_0.001 0.001 before 121 87.90 18.4088 24 97.60 50.5345
after 135 86.04 0.0355 12 98.83 39.0603
0.005 before 266 73.40 22.2503 41 95.90 93.6130
after 88 90.01 0.0341 34 96.73 24.0009
c_2 phase-flip_0.001 0.001 before 120 88.00 21.1309 24 97.60 52.8487
after 134 86.13 0.0237 12 98.83 29.8855
0.005 before 262 73.80 21.1185 41 95.90 78.1674
after 67 92.12 0.0224 25 97.60 15.3171
c_2 mixed_0.01 0.001 before 120 88.00 171.4938 24 97.60 203.8980
after 135 86.04 0.0233 12 98.83 29.9190
0.005 before 263 73.70 174.0630 41 95.90 246.1607
after 87 90.11 0.0221 35 96.64 19.6066
iris 4 c_0 noiseless 0.01 before 5 93.75 0.0020 0 100.00 9.2720
after - - - - - -
0.05 before 42 47.50 0.0009 0 100.00 137.7165
after - - - - - -
c_1 random 0.01 before 20 75.00 0.0011 0 100.00 46.0594
after - - - - - -
0.05 before 47 41.25 0.0011 0 100.00 208.1525
after - - - - - -
c_2 bit-flip_0.005 0.01 before 20 75.00 0.0028 0 100.00 46.2143
after - - - - - -
0.05 before 47 41.25 0.0031 0 100.00 208.8100
after - - - - - -
c_2 depolarizing_0.0005 0.01 before 20 75.00 0.0028 0 100.00 46.0320
after - - - - - -
0.05 before 47 41.25 0.0033 0 100.00 208.6576
after - - - - - -
c_2 phase-flip_0.005 0.01 before 20 75.00 0.0029 0 100.00 46.0522
after - - - - - -
0.05 before 47 41.25 0.0031 0 100.00 208.6859
after - - - - - -
c_2 mixed_0.0005 0.01 before 20 75.00 0.0050 0 100.00 45.7089
after - - - - - -
0.05 before 47 41.25 0.0055 0 100.00 211.5529
after - - - - - -
tfi 4 c_0 noiseless 0.1 before 558 12.81 0.0036 0 100.00 42.4537
after - - - - - -
c_1 random 0.05 before 333 47.97 0.0038 0 100.00 24.5601
after - - - - - -
0.1 before 551 13.91 0.0037 0 100.00 41.0691
after - - - - - -
c_2 bit-flip_0.02 0.05 before 354 44.69 0.0038 0 100.00 26.0638
after - - - - - -
0.1 before 571 10.78 0.0039 0 100.00 42.6273
after - - - - - -
c_2 depolarizing_0.005 0.05 before 335 47.66 0.0038 0 100.00 24.7343
after - - - - - -
0.1 before 553 13.59 0.0039 0 100.00 41.2738
after - - - - - -
c_2 phase-flip_0.0001 0.05 before 333 47.97 0.0039 0 100.00 24.5491
after - - - - - -
0.1 before 551 13.91 0.0039 0 100.00 41.0684
after - - - - - -
c_2 mixed_0.01 0.05 before 338 47.19 0.0041 0 100.00 25.0071
after - - - - - -
0.1 before 555 13.28 0.0043 0 100.00 41.4739
after - - - - - -
## Global Robustness Verification For *global-robustness*, we also add different levels of noise to each quantum model. We tested 12 QML models on *VeriQR*, covering QCNN, Quantum Approximate Optimization Algorithms (QAOA), Variational Quantum Eigensolver (VQE) and other algorithms, including: - The model $aci$, trained on the [Adult-Income dataset](https://archive.ics.uci.edu/dataset/2/adult) for income prediction. - The model $fct$, trained on a [dataset](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud) that contains credit transactions made by European cardholders, used to detect fraudulent credit card transactions. - The model $cr$, trained on the [dataset ](https://archive.ics.uci.edu/dataset/144/statlog+german+credit+data) for classifying people described by a set of attributes as good or bad credit risks. - The models $ehc$ of 6, 8, 10, 12 qubits are obtained from the experiments in [the paper](https://www.science.org/doi/abs/10.1126/science.abb9811), used for calculating the binding energy of hydrogen chains. - The model named $qaoa$ is used for hardware grid problems in [the paper](https://www.nature.com/articles/s41567-020-01105-y). - The models $iris$ and $fashion$ are same as the one in **Local-robustness Verification**. The full experimental results are shown in the [global_results](https://github.com/Veri-Q/VeriQR/blob/main/py_module/Global/results/global_results.csv) table. ### Experimental Comparison Here is an experimental comparison against a baseline implementation without tensors for global robustness verification, where "TN" and "Baseline" represent tensor-based and matrix-based implementation methods, respectively.
Model #Qubits Noise p (ε, δ) Baseline TN robust
K* time (sec.) K* time (sec.)
iris 4 bit flip 0.005 (0.003, 0.0001) 0.98859 0.01 0.98733 2.86 NO
depolarizing 0.005 (0.03, 0.0075) 0.98310 0.01 0.98077 2.48 NO
phase flip 0.0001 (0.005, 0.005) 0.99943 0 0.99938 2.39 YES
mixed 0.0001 (0.03, 0.005) 0.99968 0 0.99961 2.36 NO
ehc 6 bit flip 0.05 (0.001, 0.0005) 0.90028 0.02 0.89994 13.28 NO
depolarizing 0.075 (0.001, 0.0001) 0.90000 0.02 0.89892 13.18 NO
phase flip 0.0001 (0.005, 0.003) 1 0.02 0.99999 13.14 NO
mixed 0.01 (0.0003, 0.0005) 0.99993 0.02 0.99954 13.58 YES
ehc 8 bit flip 0.0001 (0.0003, 0.0075) 0.99980 0.26 0.99976 26.17 YES
depolarizing 0.05 (0.001, 0.0075) 0.93333 0.26 0.93304 27.87 YES
phase flip 0.025 (0.075, 0.0003) 1 0.26 0.99968 28.46 NO
mixed 0.0005 (0.005, 0.005) 0.99938 0.24 0.99905 25.75 YES
fashion 8 bit flip 0.005 (0.075, 0.005) 0.99000 0.12 0.98987 6.01 NO
depolarizing 0.025 (0.03, 0.003) 0.96274 0.16 0.96274 6.37 NO
phase flip 0.025 (0.005, 0.0003) 0.98452 0.13 0.98313 6.03 NO
mixed 0.075 (0.0005, 0.0075) 0.88444 0.14 0.88444 5.95 YES
aci 8 bit flip 0.0001 (0.003, 0.0001) 0.99985 0.18 0.99985 6.44 NO
depolarizing 0.025 (0.03, 0.0005) 0.92640 0.25 0.92440 7.70 NO
phase flip 0.05 (0.05, 0.001) 0.88450 0.19 0.85990 8.58 NO
mixed 0.005 (0.005, 0.005) 0.98384 0.22 0.98326 6.06 YES
fct 9 bit flip 0.05 (0.075, 0.003) 0.99024 0.98 0.97683 13.89 NO
depolarizing 0.05 (0.0003, 0.0001) 0.92638 0.76 0.92486 40.73 NO
phase flip 0.01 (0.01, 0.0075) 0.98730 0.87 0.98290 10.45 NO
mixed 0.05 (0.075, 0.0075) 0.94531 0.89 0.92949 9.06 NO
cr 9 bit flip 0.025 (0.01, 0.0005) 0.93964 0.65 0.93819 14.44 NO
depolarizing 0.005 (0.075, 0.005) 0.98637 1.21 0.98515 6.49 NO
phase flip 0.025 (0.0003, 0.0001) 0.94753 0.97 0.93772 9.63 NO
mixed 0.025 (0.0001, 0.0001) 0.95579 0.93 0.94980 12.15 YES
qaoa 10 bit flip 0.005 (0.05, 0.0005) 0.99843 5.23 0.98507 16.98 NO
depolarizing 0.0001 (0.01, 0.003) 0.99983 6.15 0.99965 16.10 NO
phase flip 0.005 (0.075, 0.0075) 0.99224 5.14 0.98516 17.95 NO
mixed 0.001 (0.03, 0.0075) 0.99923 4.98 0.99657 16.16 NO
ehc 10 bit flip 0.075 (0.05, 0.0003) 0.85409 3.37 0.85262 82.25 NO
depolarizing 0.0005 (0.03, 0.001) 0.99933 5.69 0.99924 40.33 NO
phase flip 0.01 (0.0003, 0.0075) 1 4.36 0.99857 66.67 YES
mixed 0.0001 (0.005, 0.001) 0.99981 5.26 0.99977 38.13 NO
ehc 12 bit flip 0.005 (0.0005, 0.0003) 0.99001 169.42 0.98965 76.77 NO
depolarizing 0.0005 (0.0001, 0.005) 0.99933 253.11 0.99926 189.35 YES
phase flip 0.075 (0.001, 0.0075) 1 163.61 0.99880 675.50 YES
mixed 0.001 (0.01, 0.0001) 0.99997 195.48 0.99984 64.50 NO
inst 16 bit flip 0.005 (0.0005, 0.0003) - TO 0.98009 1052.73 NO
depolarizing 0.0005 (0.0003, 0.005) - TO 0.99833 33.99 YES
phase flip 0.05 (0.001, 0.0075) - TO 0.95131 381.15 YES
mixed 0.001 (0.005, 0.0003) - TO 0.99899 123.25 NO
qaoa 20 bit flip 0.05 (0.005, 0.001) - TO 0.91194 2402.32 NO
depolarizing 0.075 (0.005, 0.003) - TO 0.83488 433.05 NO
phase flip 0.0005 (0.0001, 0.0001) - TO 0.99868 70.00 YES
mixed 0.05 (0.075, 0.0003) - TO 0.89682 4635.55 NO