Autoencoder-Based Efficient Feature Extraction Method for Position Invariant Image
Date Issued
2024
Author(s)
Naghizade, Elshan
Abstract
This master’s thesis explores an effective method for extracting features from images that do
not change position, using a technique based on autoencoders. The focus is on using
Convolutional Autoencoders (CAEs) and Variational Autoencoders (VAEs) to process two
specific types of data: the MNIST dataset, which includes images of handwritten digits, and
the Bristol-Myers Squibb – Molecular Translation dataset, which contains images of hand drawn chemical structures.
The approach involves training both CAE and VAE models on these datasets to create latent
vectors—compressed representations that summarize key aspects of the images. For the
MNIST dataset, these vectors are used to train deep neural networks (DNNs) for classifying
the digits. The results show high accuracy, with CAE-based DNNs achieving 96% and VAE based DNNs achieving 91% accuracy on the training set. The performance is further
analyzed for each digit, using bar graphs and confusion matrices to demonstrate that CAEs
provide more accurate and consistent classifications.
For the chemical structure images in the Bristol-Myers Squibb dataset, the thesis tests three
different setups for converting images into textual formulas. These setups include a
combination of EfficientNet, Vision Transformer (ViT), and a traditional Transformer model,
as well as pipelines using features from VAEs and CAEs paired with ViT and Transformer.
The effectiveness of these models is measured using the Levenshtein distance, a metric that
quantifies the difference between the predicted text and the actual formula. The CAE-based
model outperforms the others, showing it can more accurately translate images into text.
The study demonstrates the value of using features specific to the data being analyzed. It
shows that CAE and VAE models, when trained on specific datasets like the Bristol-Myers
Squibb, can capture essential details better than more general models. The thesis wraps up by
suggesting directions for future research, such as implementing more advanced data
augmentation techniques, applying transfer learning from specialized fields, exploring
different neural network structures, testing the models on various datasets, and finding ways
to make training and using these models more efficient.
not change position, using a technique based on autoencoders. The focus is on using
Convolutional Autoencoders (CAEs) and Variational Autoencoders (VAEs) to process two
specific types of data: the MNIST dataset, which includes images of handwritten digits, and
the Bristol-Myers Squibb – Molecular Translation dataset, which contains images of hand drawn chemical structures.
The approach involves training both CAE and VAE models on these datasets to create latent
vectors—compressed representations that summarize key aspects of the images. For the
MNIST dataset, these vectors are used to train deep neural networks (DNNs) for classifying
the digits. The results show high accuracy, with CAE-based DNNs achieving 96% and VAE based DNNs achieving 91% accuracy on the training set. The performance is further
analyzed for each digit, using bar graphs and confusion matrices to demonstrate that CAEs
provide more accurate and consistent classifications.
For the chemical structure images in the Bristol-Myers Squibb dataset, the thesis tests three
different setups for converting images into textual formulas. These setups include a
combination of EfficientNet, Vision Transformer (ViT), and a traditional Transformer model,
as well as pipelines using features from VAEs and CAEs paired with ViT and Transformer.
The effectiveness of these models is measured using the Levenshtein distance, a metric that
quantifies the difference between the predicted text and the actual formula. The CAE-based
model outperforms the others, showing it can more accurately translate images into text.
The study demonstrates the value of using features specific to the data being analyzed. It
shows that CAE and VAE models, when trained on specific datasets like the Bristol-Myers
Squibb, can capture essential details better than more general models. The thesis wraps up by
suggesting directions for future research, such as implementing more advanced data
augmentation techniques, applying transfer learning from specialized fields, exploring
different neural network structures, testing the models on various datasets, and finding ways
to make training and using these models more efficient.