TechKnowHow Club

Serial Port for emulated esp32 using QEMU(?)

Aryan — Mon, 01 Sep 2025 09:29:16 GMT

Previous posts walk through how to run code on an esp32 emulator using QEMU and how to create virtualized sensors for esp32 emulator. This post explains how you can set this up in a way that external applications such as c++/python/nodejs apps can read data from the esp32 emulator. Building IoT controllers and dashboards requires the ability of the app to be able to read data from the SerialPort. This becomes tricky when using emulators - not only you will need the emulator to process data like the real hardware, you will also need to make sure the emulator and software are able to integrate and communicate. When working with hardware you would print data over the serial port using something like Serial.println(data); and read it in JS using something like const { SerialPort } = require('serialport'); const port = new SerialPort({ path: '/dev/ttyUSB0', baudRate: 9600, }); port.on('open', () => { console.log('Serial port opened'); }); port.on('data', (data) => { console.log('Received:', data.toString()); }); port.on('error', (err) => { console.error('Serial port error:', err.message); }); Now how would we do this with an emulator running on QEMU? Modify QEMU command - Update your QEMU command to expose the serial port via TCP. This makes QEMU listen on TCP port 1234 for serial connections. qemu-system-xtensa -nographic -machine esp32 \ -drive file=flash_image.bin,if=mtd,format=raw \ -serial tcp::1234,server,nowait Create a Node.js script to read from the TCP port: const net = require('net'); const client = net.createConnection({ port: 1234 }, () => { console.log('Connected to ESP32 emulator'); }); client.on('data', (data) => { console.log(data.toString()); }); client.on('error', (err) => { console.error('Connection error:', err); }); client.on('close', () => { console.log('Connection closed'); }); now you should be able to run node reader/read.js to get the data being printed on the serial port. Code to this project can be found on this GitHub repo. This is part 3 of a project of me trying to develop a tool which will make it much easier for developing softwares which need to be integrated with embedded devices.

Virtualized sensors for esp32 emulator

Aryan — Sat, 23 Aug 2025 14:22:15 GMT

Okay so you've got your esp32 emulator running on qemu. What next? I need to virtualize sensors. That cannot be done really because sensors return environmental parameters, so you can either simulate environmental parameters or generate mock data instead. For context I would suggest to read the previous post. I started off simple trying to generate mock data for the bmp280 sensor. Now to have your firmware behave with the virtualized sensor to be exactly like the real sensor you would have to create a new c++ library with the same classes, interfaces, and functions which take in the same parameters, and give out the same outputs so that there are no problems in migrating from testing it on real hardware and the emulator. The goal is to be able to run the same code on the emulator without changing it. For this the most appropriate way would be to find the source code for the c++ library of the sensor that you would be using. In my case that would be the Adafruit_BMP280.h and replicate it entirely, replacing the data it returns from the real hardware with any data you'd like. You can save a lot of time by just implementing a couple of commonly used functions that you are using in your code. A lot of these libraries are pretty big and repeating this for every library would be very time consuming. Here is what i did // utils/sensors/bmp280.hpp #ifndef BMP280_HPP #define BMP280_HPP #include #include "freertos/FreeRTOS.h" #include "freertos/task.h" #ifdef __cplusplus extern "C" { #endif class bmp280 { public: bmp280(); bool begin(uint8_t addr = 0x77, uint8_t chipid = 0x58); void bmp280_init(); float readPressure(); float readTemperature(); float readAltitude(float seaLevelhPa = 1013.25); }; #ifdef __cplusplus } #endif #endif // utils/sensors/bmp280.cpp #include "bmp280.hpp" #include #include static float time = 0.0; bmp280::bmp280() {} bool bmp280::begin(uint8_t addr, uint8_t chipid) { vTaskDelay(pdMS_TO_TICKS(200)); return true; } void bmp280::bmp280_init() { printf("BMP280 Initialized\n"); } float bmp280::readPressure() { float pressure = 1013.25 + 10 * sin(time); time += 0.1; return pressure; } float bmp280::readTemperature() { float temperature = 25.0 + 5 * sin(time); return temperature; } float bmp280::readAltitude(float seaLevelhPa) { float altitude = 44330 * (1 - pow(readPressure() / seaLevelhPa, 0.1903)); return altitude; } And this can be used in your main/main.cpp like so // main/main.cpp #include #include "freertos/FreeRTOS.h" #include "freertos/task.h" #include "bmp280.hpp" extern "C" void app_main() { bmp280 bmp; if (!bmp.begin()) { printf("Failed to initialize BMP280\n"); return; } printf("BMP280 Ready!\n"); while (1) { float temperature = bmp.readTemperature(); printf("Temperature = %.2f *C\n", temperature); float pressure = bmp.readPressure(); printf("Pressure = %.2f hPa\n", pressure); float altitude = bmp.readAltitude(); printf("Altitude = %.2f m\n", altitude); vTaskDelay(1000 / portTICK_PERIOD_MS); } } Be sure to add the dependancies in main/CMakeLists.txtlike so idf_component_register(SRCS "main.cpp" "../utils/sensors/bmp280.cpp" INCLUDE_DIRS "../utils/sensors/") This is a very simple demonstation which is actually very intuitive. I plan on creating libraries like such for all the sensors and data storage modules I will be needing. Then the plan is to figure out a way to write tests and have the data returned by these sensor libraries to main/main.cpp based on the tests. So I would be able to write the tests once which would include multiple scenarios and edge cases and just run a command to see if the firmware I have written passes the test cases on the emulator itself without actually going and performing the tests with the real hardware - saving me a lot of time.

esp32 on QEMU

Aryan — Wed, 20 Aug 2025 16:22:05 GMT

Not having the entire hardware for embedded projects is a pain. It stops the development entirely and is extremely inefficient. In traditional software dev - development, testing and deployment are 3 entirely different phases which are mostly independant of each other. Eventhough software development methodologies such as iterative development, agile development, rapid prototyping exist however the waterfall model is intuitive and is commonly used while building projects. Embedded development (atleast for non-industry grade projects) requires development and testing to happen side by side to ensure what you're building works! Now you cannot really always have the hardware on hand, which makes development tricky. We faced a similar problem during the development of our flight software for our newest rocket Airavata at Team Sammard. The development timelines of the flight computer and flight software run paralelly which means for a majority chunk of the time we do not have access to the hardware on which our code will run. This means we can write our code but not really test it. I took this to reddit to search for solutions where I was suggested to simply buy the components and make the flight computer on a breadboard and be done with it. One can argue that the components do not cost that much to begin with and buying them to save time would make sense. However since I come from a pure software background I wanted a solution which not only solved my current problem but also gave me more. The Plan! So the plan was to create a custom flight computer emulator, which would behave exactly how the real flight computer would behave. This means that not only the microcontroller will be virtualized but also the sensors, storage, actuators, etc. In theory since we are virtualizing the sensors we can have them return any data we'd like which means we can write tests and ahve the sensors return data based on the tests which will allow us to see how oour flight software is performing in those situations. This particular approach is better than building the flight computer on a breadboard as here we can test the behaviour of the flight software in situations such as reaching the highest point in flight to check for parachute ejection. Which is a very risky test to perform with real hardware as it has the risk of damaging the hardware. Prerequisites `I will strongly recommend using Linux for this. Not that this is not possible on Windows but it is just far more easier on linux. 1. ESP-IDF Our flight computer is powered by the esp32 by espressif. Run the following commands to isntall and setup ESP-IDF. mkdir -p ~/esp cd ~/esp git clone --recursive https://github.com/espressif/esp-idf.git cd esp-idf ./install.sh This will install all necessary tools. 2. Set Up Environment Variables Once the installation is complete, you need to load the environment variables: source ~/esp/esp-idf/export.sh Now try running: idf.py --version 3. Make It Permanent To avoid running source ~/esp/esp-idf/export.sh every time, add it to your shell configuration: echo 'source ~/esp/esp-idf/export.sh' >> ~/.bashrc Then restart your terminal and check again. 4. Install Required Dependancies Before installing QEMU, install the necessary system libraries: sudo apt-get install -y libgcrypt20 libglib2.0-0 libpixman-1-0 libsdl2-2.0-0 libslirp0 2. Install the ESP32 QEMU Emulator Run the following command to install the pre-built QEMU binaries: python $IDF_PATH/tools/idf_tools.py install qemu-xtensa qemu-riscv32 If $IDF_PATH is not set, try: python ~/esp/esp-idf/tools/idf_tools.py install qemu-xtensa qemu-riscv32 After installation, make sure QEMU is added to your PATH: source ~/esp/esp-idf/export.sh To verify, check the installed tools: python $IDF_PATH/tools/idf_tools.py list You should see qemu-xtensa and qemu-riscv32 in the list. Run a Sample ESP32 Application in QEMU Now that QEMU is installed, let's run a test application: 1. Create a New ESP-IDF Project cd ~/esp idf.py create-project hello_world cd hello_world 2. Configure the Project for QEMU idf.py set-target esp32 3. Write Sample Code Replace the default main.c with the following code. Create or modify the file main/main.c: #include #include "freertos/FreeRTOS.h" #include "freertos/task.h" void hello_task(void *pvParameter) { while(1) { printf("Hello, World!\n"); vTaskDelay(pdMS_TO_TICKS(1000)); // Wait for 1 second } } void app_main() { xTaskCreate(&hello_task, "hello_task", 2048, NULL, 1, NULL); } 4. Build the Project Compile the project: idf.py build This will generate the required firmware files. 5. Run on QEMU Merge binaries: esptool.py --chip esp32 merge_bin -o flash_image.bin --flash_mode dio --flash_size 4MB \ 0x1000 build/bootloader/bootloader.bin \ 0x8000 build/partition_table/partition-table.bin \ 0x10000 build/esp32_counter.bin Run in QEMU:\ qemu-system-xtensa -nographic -machine esp32 \ -drive file=flash_image.bin,if=mtd,format=raw In case your flash_image.bin is less than 4.0MB you will have to run: truncate -s 4M flash_image.bin verify the size of flash_image.bin using: ls -lh flash_image.bin Then re-run using QEMU. I am currently in the works of developing a custom emulator to test our flight software. The repository for which can be found here on GitHub. In case you're interested in something similar or found this cool do drop a comment below.

BitFlash_Client - Open Source FOTA for Everyone

Aryan — Tue, 11 Feb 2025 00:27:20 GMT

Last week Prakriti & I created BitFlash_Client which is now officially a part of the Arduino Library Registry. You can try out the library and let us know if you liked it and if you have any feedback we'd love to hear you out. Also if you're interested in contributing to the project we're up for that too (we are planning on continuing this project past the hackathon stage). So what is BitFlash_Client? BitFlash_Client allows embedded devices to check for firmware updates in the background and update and reboot themselves once a new firmware is detected. All this is done by checking a hosted .json file for new updates. Flashing code again and again onto an embedded device is a pain and fellow IoT developers will agree that not only is it tiresome but when you're working with a network of these devices and you have to change the firmware running on multiple devices it really just gets very very annoying. What is FOTA? FOTA - Firmware Over The Air is a technology is the underlying concept of BitFlash_Client which allows IoT devices to download new firmware and reflash the new firmware onto themselves wirelessly without human intervension. Mostly WiFi is used for downloading firmware updates as radio communication doesn't have enough bandwidth to support firmware transfers in a realistic scenario. What are the features of BitFlash_Client? All firmwares can be updated by a single git push as the binaries are hosted on the user's github itself. You can deploy multiple firmwares at once. You can set the delay between 2 consecutive checks for new firmware. BitFlash_Client started out as a hackathon project and for testing and deployment we needed to see if the embedded device (in our case an ESP32) was able to check for updates on a hosted .json file, if yes then the .json file would point towards the url from where the device could download the binary for the firmware. Now this means 2 things need to be hosted here: .json file - where you will check for the version of the firmware .bin file - from where you will download the firmware Since this is a solution targets at making things simpler we decided to offer a combined solution - An electronjs app for managing and monitoring the devices, a webserver hosting the .json file, and BitFlash_Client. We were struck with one question - where to host the compiled binaries? Because if we we're to host them on the same server as the .json file it would mean we would first have to bring the binaries from the user's github repository after every git push. This seemed too complicated of a solution which we wouldn't have time to complete so we opted for a cheeky solution - let the binaries be hosted on the user's github itself and simply point to the repo from the .json file. Not only did this save us a lot of time but also made sure that binaries would be instantly available as soon as a git push was performed. So what's left? Unfortunately we were unable to finish all the features we had planned out but we are going to finish them soon. Perform shell scripting in the electronjs app to compile all binaries from code automatically. Minor bug fixes in BitFlash_Client Configure github auth to allow private repo access.

Object Detection - The Basics.

Krish — Wed, 27 Nov 2024 14:55:00 GMT

What is object recognition? Before we delve into that, how do we identify images? In a general sense, an object is any item, entity, or thing that can be perceived, identified, or defined. Objects have identifiable characteristics, such as shape, size, and color, which make them distinct from other objects. IT IS THIS SET OF CHARACTERISTICS THAT IS EMPLOYED IN OBJECT RECOGNITION. In the context of Artificial Intelligence (AI), particularly in computer vision, an object is a specific, identifiable part of an image or scene that the AI aims to recognize and classify. For example, in an image, objects could include things like a car, a person, a building, or a tree. Object recognition is the process by which an AI system detects these items in visual data, such as photos or videos, and identifies what they are. In object recognition, the AI model processes visual data, identifies specific patterns and features (like edges, shapes, or textures), and classifies them as particular types of objects. Through training on large datasets, the model learns to recognize and label these objects across various scenes, angles, and conditions. Healthcare: Object recognition helps analyze medical images to detect issues like tumors, improving diagnostic precision, and assists in patient monitoring to enhance safety. Surveillance: It enables real-time detection of suspicious activities and unauthorized access, strengthening security, and aids in identity recognition for access control. Autonomous Vehicles: Object recognition allows vehicles to detect pedestrians, road signs, and obstacles, enabling safe navigation and real-time decision-making in traffic. Retail: It supports automated checkout by identifying items, streamlining the shopping experience, and improves inventory management by tracking stock on shelves. Agriculture: Object recognition identifies crop health issues and weeds for precision farming, helping conserve resources and improve yield, and supports livestock monitoring. Smart search engines Object recognition enhances smart search engines by enabling visual search capabilities, where users can search with images rather than text. This technology recognizes objects within images and provides contextually relevant results, making searches more intuitive and accurate. For example, a user can upload a photo of a product, and the search engine identifies similar items or details. It also supports filtering results based on visual characteristics, improving the relevance of search outputs. Additionally, it helps identify landmarks, plants, or animals in images, offering useful information for educational or travel purposes. Object recognition in smart search engines transforms browsing by making image-based searches more intuitive, allowing users to find information or products faster without needing precise keywords. In marketing, it personalizes user experiences by suggesting visually similar items, helping brands reach target audiences with relevant products. It enhances overall productivity by simplifying information retrieval, enabling faster decision-making and reducing time spent on searches. This technology also supports content creators and businesses by auto-tagging images and improving asset management. Ultimately, it aligns browsing, marketing, and productivity with a more visual, user-centric approach. Services enabled by smart search engines Google Lens uses object recognition to identify objects, text, and scenes in real-time through a smartphone camera, offering diverse services. It can translate text instantly by recognizing languages and overlaying translations on the screen, making it useful for travelers. Lens also provides shopping assistance by identifying products and suggesting similar items available online. Additionally, it recognizes plants, animals, and landmarks, offering educational insights or travel information. By turning a camera into an interactive tool, Google Lens makes everyday tasks like language translation, shopping, and learning more accessible and efficient. Convolutional Neural Networks (CNN) A Convolutional Neural Network (CNN) is a type of artificial intelligence model inspired by how the human brain processes visual information. It’s specifically designed to analyze data with a grid-like structure, like images or audio. The CNN works by breaking down an image into small, manageable parts called "filters" or "kernels." These filters slide over the image, focusing on small sections at a time. This process helps the network detect simple patterns, like edges or colors, in the early layers. As the information moves through deeper layers of the network, the CNN starts recognizing more complex patterns by combining the simpler ones, eventually forming a high-level understanding of what’s in the data. Through many layers of filtering and pattern recognition, CNNs learn to identify features without requiring specific programming for each detail. This makes them great at spotting patterns in any type of visual or structured data, like sorting handwritten numbers, detecting textures, or analyzing medical scans. A Convolutional Neural Network is like a digital artist that learns to see the world, recognizing patterns in pixels to transform raw data into understanding, one layer at a time. Deep Learning & CNN Deep learning is like teaching a computer by showing it tons of examples until it learns to make decisions on its own. It’s like when you learn what animals look like by seeing lots of pictures of them – after a while, you can tell a dog from a cat just by looking. Deep learning is a type of AI where computers learn patterns by processing huge amounts of data through many layers, each layer building on the last. When combined with CNNs, deep learning helps computers analyze images by finding patterns in small parts first and then understanding the whole picture. Each layer in a CNN adds to this learning process, making the system more accurate and able to understand complex details. Deep learning with CNNs is like teaching a computer to see, layer by layer, until it understands images the way we do, from simple edges to complex objects. Deep learning is the art, and CNNs are the chisel—precision tools that carve raw data into clear vision. Bias, data integrity and completeness Object detection models can inherit biases from the training data, which may lead to inaccuracies or unfair predictions, especially if certain classes or demographics are underrepresented. For instance, if a dataset contains more images of certain objects or ethnicities, the model may perform better on those but poorly on others. This bias can have significant real-world implications, particularly in areas like security, healthcare, and autonomous driving, where fair and unbiased recognition is crucial. To address bias in CNNs, diversify the training data to include various demographics and scenarios. Use data augmentation (flipping, rotating) to expand dataset diversity. Apply fairness-aware algorithms that re-weight classes or penalize biased errors. Transfer learning from a pre-trained, diverse model can also help reduce bias. Continuously evaluate the model across different groups and involve a diverse team to spot potential biases early. Models - YOLO YOLO (You Only Look Once) is a fast and efficient object detection model designed for real-time applications. Unlike traditional models that apply sliding windows or region proposals, YOLO treats object detection as a single regression problem, predicting bounding boxes and class probabilities directly from the full image in one pass. This approach allows YOLO to detect multiple objects in an image with high speed, making it ideal for applications requiring real-time detection, such as autonomous driving, surveillance, and robotics. YOLO’s architecture divides the image into a grid, where each cell predicts a set number of bounding boxes and confidence scores for object classes.. Additionally, by using a single neural network, YOLO avoids redundant computations, making it far more efficient than earlier region-based methods like R-CNN. Models - resnet ResNet (Residual Network) is a deep learning architecture designed to solve the problem of training very deep neural networks, which often suffer from vanishing gradients and accuracy degradation as layers increase. ResNet introduced residual connections, or “skip connections,” that allow the model to bypass certain layers. This innovation enables the network to pass information forward even if certain layers don’t contribute, making it possible to train networks with hundreds of layers effectively. each set of layers learns only the “residual” (or difference) between the input and the desired output rather than learning the full transformation. Mathematically, each residual block learns a function F(x) + x , where F(x) represents the transformation learned by the layers and x is the input passed directly to the output. This structure helps prevent gradient issues by maintaining a direct flow of information and gradients throughout the network. Models - VGG VGG-19 is a deep convolutional neural network (CNN) developed by the Visual Geometry Group (VGG) at the University of Oxford, introduced in 2014. Known for its simplicity and depth, VGG-19 contains 19 weight layers and performs exceptionally well on large-scale image recognition tasks like ImageNet. Convolutional Layers: 16 convolutional layers using 3x3 filters with stride 1 and padding 1, preserving spatial resolution. Activation Function: ReLU activation applied after each convolution layer to add non-linearity. Pooling Layers: Max-pooling (2x2 filters with stride 2) used after each convolution block to reduce spatial dimensions. Fully Connected Layers: 3 fully connected layers at the end for classification. Softmax Layer: For outputting class probabilities in multi-class tasks (e.g., 1000-class ImageNet classification). Impact on Object Detection: Feature Extraction: The deep layers of VGG-19 are effective at extracting intricate features from images, making it highly useful for object detection tasks. Transfer Learning: Pre-trained VGG-19 models are widely used in transfer learning for object detection, where they are fine-tuned on domain-specific datasets. Influence on Modern Models: VGG-19's architecture influenced models like ResNet and Inception, particularly its deep architecture and consistent use of small convolutional filters. Why It's Important: Simplicity & Performance: The uniform structure of VGG-19, with its focus on small 3x3 filters, enables powerful feature learning that has become foundational in modern computer vision systems. Recent Trends Improvements Accuracy Improvements: New architectures, like EfficientNet and ResNet, continue to push the boundaries in image recognition accuracy. Self-Supervised Learning: Techniques that reduce dependency on large labeled datasets are emerging, potentially allowing models to learn from less annotated data, which saves time and cost. Optimisation Lightweight models like MobileNet and EfficientNet-Lite are optimized for devices with limited processing power, like smartphones. They maintain good accuracy while reducing size, memory, and computation needs, enabling real-time performance on mobile devices. These models use depthwise separable convolutions, breaking down standard convolutions into simpler steps to reduce parameters and computation. They also apply pruning (removing unnecessary weights) and quantization (using lower-precision data types) to cut down memory use. EfficientNet further optimizes by scaling depth, width, and resolution, balancing accuracy with efficiency for compact, resource-friendly deployments.

Setting Up HTTPS on IIS in 2024

Aryan — Sun, 27 Oct 2024 13:26:14 GMT

Hosting websites on a windows server using the IIS - Internet Information Services is painful in 2024 for a new developer. The technology is so old that people who know how to use this are very proficient in it and the documentation is very limited and assumes you know a lot of the stuff beforehand which makes it difficult for new developers like me to actually get the stuff working without hitting a roadblock. One such issue is setting up https for your website by binding your SSL certificates. SSL certificates prove that the website you’re connecting to genuinely owns the registered domain name, helping ensure secure, encrypted communication between the user and the correct server. When my previous SSL certificate expired I went and bought the same SSL i had earlier without skipping a beat. But you might want to deciede what type of SSL certificate you should purchase based on your future requirements. The most common options are: Single-Domain SSL: Secures a single domain (e.g., techknowhow.club). Wildcard SSL: Secures a domain and all its subdomains (e.g., *.techknowhow.club). Multi-Domain (SAN) SSL: Secures multiple specified domains or subdomains in one certificate. I did not plan on this which is why i ran into a problem shortly after setting up the SSL for techknowhow.club. Let us try understanding how to actually do this now. Firstly we need to generate a CSR code on our IIS server which will be used to create the SSL certificate. You can create the SSL certificate by generating a CSR code from the SSL provider directly also but i would not recommend that approach. Generating CSR code on iis server You need to go to the server panel in your iis and go to server certificates. Create a Certificate Request from the top right panel. Enter your details, select 2048 bits for the encryption and select the path where you want to create a certificate request and name the file. On completion a .txt file will be created at the defined location whose contents look somewhat like this. -----BEGIN CERTIFICATE REQUEST----- . . . . . . . -----END CERTIFICATE REQUEST----- Copy the entire contents of this file and use this as the CSR code while creating the SSL certificate from any SSL provider like GoGetSSL or ZeroSSL. Validate your SSL Certificate Now your SSL certificate will be generated but you still need to validate that the domain name also belongs to you. There are multiple ways you can choose to validate this the easiest i find is to use the validate using DNS option. Validate using create a DNS record You will be provided a DNS record by the SSL provider which you need to create through the service which handles your domain/dns. For me its godaddy so i simply log on to godaddy and create a DNS record. Generally you would have to create a DNS record of the type CNAME with the given target and host. You are not required to enter your domain name anywhere. Also feel free to keep a custom TTL (time to live) - i keep mine at 600 seconds which is the lowest godaddy allows. Once this is done validate the DNS record from your SSL provider's portal, and you should have your SSL certificate in minutes. Installing the Certificate Now add this certificate to the same path which you defined while creating the certificate request. Make sure you do not move the files around. I had the genius idea of organizing my old certificate files and new certificate files in 2 different folders after creating the certificate request which meant when i wanted to complete the certificate request my path wasnt the same - which caused iis to reject my certificate every time. So the way this works is that when you create a certificate request through iis it creates: CSR code Private key The private key is stored on the server and is associated with the CSR code and any request with the path and same CSR code. Now we need to complete the certificate request for which you need to move your SSL certificate to the same folder with your CSR code and through iis - Complete Certificate Request. If the CSR code doesn't match with the private key or the SSl does not match with the CSR code the certificate will be rejected. Binding the SSL Certificate to your website Now choose your website from the section on the left on iis and click on bindings. If you already have https bindings edit them or create a new binding using https make sure you select the same SSL certificate here which you have just created. I would recommend deleting the old certificates from iis to avoid confusion. Simply restart your server once this is done and you should be good to go! My Mistake After configuring SSL for techknowhow.club i went and tried to add the same certificate for my subdomains like codeclip.techknowhow.club which is when i realised that you need a different type of SSL certificate altogether for this; and those are expensive. Another alternative to the expensive multidomain r wildcard SSL certificates would probably be to just buy another domain SSL certificate. This works only if you have only a few subdomains otherwise this would become more expensive than buying a multidomain or wildcard. Generally domain SSLs are cheaper than the other options. Another option is to use a free SSL certificate however those have a much shorter validity period of only 90 days. But they might just be a really good option for maybe a hackathon project or a capstone project.

Building CodeClip: Copy Your Entire Codebase Instantly

Aryan — Sat, 26 Oct 2024 18:56:52 GMT

As developers, we’ve all been there: you need to share either parts of your code or sometimes your entire codebase with LLMs. The inspiration was drawn from a linkedin post I saw which showcased a tool which can copy an entire github repository which you can give to LLMs, it had a UI and everything where you could select which files you wanted to include/exclude. This got me thinking this actually seems useful and I might actually use this. Which is when I realized that most of the time I want to give my code as context rather than code which is in a repository as we commit only after completion of a fix/feature and not while it is in development. This means I need a tool which will copy everything in my working directory. There's a pretty good chance that something like this probably exists but implementing a filepath.walk() felt easier than actually looking up if this actually exists. Which is why I decided to make this anyways. You can find the code to CodeClip here. I began by laying out the requirements, which look somewhat like this: Should be lightweight and quick. Shouldn't take too much time or effort to use. Should copy everything to clipboard. Should copy contents to a file called codebase_dump.txt After which i started implementing it and i tested it out on a mern project i had running locally. Which is when i ran into some error. I tried to run it in the codeclip directory itself where it worked :| Okay so that was when i realised that the error might've originated out of codeclip trying to read all files from node_modules and hit a limit either while copying or writing the contents. Soooo we need this tool to only copy files where your code lies. I implemented this just by adding a blacklist of directories and file extensions to avoid. I will be adding support for users to configure it themselves and also copy only files with certain extensions such as cclip -f .js .py .json. I will also be adding support for users to be able to choose if they want to save it to clipboard or save it to a .txt file or both. This was just something i thought would be cool to make and something i will genuinely use. You can find the code here, and if you want to try out codeclip click here. If you have any suggestions or any feedback let me know in the comments, i would love your opinion on this, maybe we can collaborate if you're interested on a particular feature you'd like to see in CodeClip.

Seamless User Management: Integrating Firebase Authentication with Odoo

Akshit — Sat, 14 Sep 2024 17:54:13 GMT

Introduction In this article, we'll explore how to integrate Firebase Authentication with Odoo ERP, creating a seamless connection between user management in Firebase and partner management in Odoo. This integration allows you to store Firebase UIDs in the Odoo model and link user actions between both systems. Prerequisites: Node.js and npm installed Firebase Project setup Odoo running instance Step 1: Setting up the project npm init -y npm install express firebase dotenv xmlrpc Step 2: Setup Firebase Configuration Create a firebaseConfig.js file to initialize the Firebase client SDK import { initializeApp as initializeClientApp } from "firebase/app"; import dotenv from 'dotenv'; dotenv.config(); const firebaseConfig = { apiKey: process.env.API_KEY, authDomain: process.env.AUTH_DOMAIN, projectId: process.env.PROJECT_ID, storageBucket: process.env.STORAGE_BUCKET, messagingSenderId: process.env.MESSAGING_SENDER_ID, appId: process.env.APP_ID, measurementId: process.env.MEASUREMENT_ID }; const firebaseClientApp = initializeClientApp(firebaseConfig); export default firebaseClientApp; Step 3: Odoo Configuration Create an odooConfig.js file to configure odoo connection import dotenv from 'dotenv'; import xmlrpc from 'xmlrpc'; dotenv.config(); const ODOO_URL = process.env.ODOO_URL; const ODOO_DB = process.env.ODOO_DB; const ODOO_USERNAME = process.env.ODOO_USERNAME; const ODOO_PASSWORD = process.env.ODOO_PASSWORD; async function getOdooConnection() { return new Promise((resolve, reject) => { const common = xmlrpc.createClient({ url: `${ODOO_URL}/xmlrpc/2/common` }); common.methodCall('authenticate', [ODOO_DB, ODOO_USERNAME, ODOO_PASSWORD, {}], (err, uid) => { if (err || !uid) { return reject('Authentication failed'); } console.log(`Authenticated with UID: ${uid}`); const models = xmlrpc.createClient({ url: `${ODOO_URL}/xmlrpc/2/object` }); resolve({ models, uid }); }); }); } export { ODOO_URL, ODOO_DB, ODOO_USERNAME, ODOO_PASSWORD, getOdooConnection }; Step 4: Creating Authentication Routes Create a authRoute.js file to handle signup, signin, and signout operations import express from 'express'; import { getAuth, signInWithEmailAndPassword, createUserWithEmailAndPassword, signOut } from 'firebase/auth'; import firebaseClientApp from './firebaseConfig.js'; import { getOdooConnection, ODOO_DB, ODOO_PASSWORD } from './odooConfig.js'; const clientAuth = getAuth(firebaseClientApp); const router = express.Router(); router.post('/signup', async (req, res) => { const { email, password } = req.body; try { const { models, uid } = await getOdooConnection(); // Create user in Firebase const userCredential = await createUserWithEmailAndPassword(clientAuth, email, password); const firebaseUid = userCredential.user.uid; // Check if user already exists in Odoo const existingPartnerId = await new Promise((resolve, reject) => { models.methodCall('execute_kw', [ODOO_DB, uid, ODOO_PASSWORD, 'res.partner', 'search', [[['x_firebase_uid', '=', firebaseUid]]]], (error, result) => { if (error) { reject(error); } else { resolve(result[0]); } }); }); let odooPartnerId = existingPartnerId; if (!existingPartnerId) { // Create new partner in Odoo odooPartnerId = await new Promise((resolve, reject) => { models.methodCall('execute_kw', [ODOO_DB, uid, ODOO_PASSWORD, 'res.partner', 'create', [{ 'name': email, 'email': email, 'x_firebase_uid': firebaseUid }]], (error, result) => { if (error) { reject(error); } else { resolve(result); } }); }); } res.status(200).json({ message: 'User signed up successfully', uid: firebaseUid, odooPartnerId }); } catch (error) { console.error('Error during signup:', error); res.status(500).json({ message: 'Failed to sign up user' }); } }); router.post('/signin', async (req, res) => { const { email, password } = req.body; try { const userCredential = await signInWithEmailAndPassword(clientAuth, email, password); const firebaseUid = userCredential.user.uid; const { models, uid } = await getOdooConnection(); // Find corresponding partner in Odoo const partnerId = await new Promise((resolve, reject) => { models.methodCall('execute_kw', [ODOO_DB, uid, ODOO_PASSWORD, 'res.partner', 'search', [[['x_firebase_uid', '=', firebaseUid]]]], (error, result) => { if (error) { reject(error); } else { resolve(result[0]); } }); }); if (!partnerId) { throw new Error('No corresponding partner found in Odoo'); } res.status(200).send({ status: 'success', user: userCredential.user, partnerId }); } catch (error) { res.status(400).send({ status: 'error', message: error.message }); } }); router.post('/signout', async (req, res) => { try { await signOut(clientAuth); res.status(200).send({ status: 'success', message: 'User signed out successfully' }); } catch (error) { res.status(400).send({ status: 'error', message: error.message }); } }); export default router; Step 5: Setting Up Odoo Log in to your Odoo instance as an administrator. Go to Settings > Technical > Database Structure > Models. Find the "res.partner" model and add a new field: Field Name: x_firebase_uid Field Label: Firebase UID Field Type: Char This will allow us to store the Firebase UID for each user in Odoo. Step 6: Create the main server Create a server.js file to setup your express application Copyimport express from 'express'; import authRoutes from './authRoutes.js'; const app = express(); app.use(express.json()); app.use('/auth', authRoutes); const PORT = process.env.PORT || 3000; app.listen(PORT, () => console.log(`Server running on port ${PORT}`)); Step 7: Environment Variables Create a .env file in your project root to store sensitive information: API_KEY=your_firebase_api_key AUTH_DOMAIN=your_firebase_auth_domain PROJECT_ID=your_firebase_project_id STORAGE_BUCKET=your_firebase_storage_bucket MESSAGING_SENDER_ID=your_firebase_messaging_sender_id APP_ID=your_firebase_app_id MEASUREMENT_ID=your_firebase_measurement_id ODOO_URL=your_odoo_url ODOO_DB=your_odoo_database_name ODOO_USERNAME=your_odoo_username ODOO_PASSWORD=your_odoo_password By following these steps, you've successfully integrated Firebase Authentication with Odoo ERP. This seamless connection enables you to manage users in Firebase while maintaining partner data in Odoo. Future improvements could include extending this integration to other Firebase services or adding additional functionality in Odoo based on user actions. Something not working? Ensure that your environment variables are correctly configured and that the Odoo instance is properly connected. Check Firebase project permissions and Odoo server logs for detailed error information. Still not working? Comment below and i'll make sure I get back to you!

Captive Portals: How to automate them

Devansh Agarwal — Sat, 03 Aug 2024 17:16:55 GMT

What are Captive Portals? If you have ever been to a hotel which provides wifi facilities, when you tried to connect to it, you would be redirected to a web page. This web page is known as Captive Portal. Captive Portals are used in various public and semi-public spaces for many reasons like access control or usage tracking or just enhancing user experience with fancy looking pages. Why you may need to automate the process of login? Some places store the authentication results in persistent storage cookies meaning you will not have to login using captive portal again and again but some places do not use this mechanism. In these places, you are constantly pestered with these captive portals which gets boring after a point. To provide a solution to this problem, I bring to you a solution which could solve your problems. Prerequisites Python (anything >= 3.7 should work) selenium (a python package) WebDriver (corresponding to the browser you want to use) Below are the resources where you can download web drivers from - 1. Firefox 2. Microsoft Edge 3. Opera 4. Chrome (not recommend since it is buggy) Once you have installed the resources, we can proceed. The Solution - The Python Script - from selenium import webdriver from selenium.webdriver.firefox.service import Service from time import sleep from selenium.webdriver.common.by import By geckodriver_path = "/path/to/your/webdriver" service = Service(executable_path=geckodriver_path) driver = webdriver.Firefox(service=service) try: # Navigate to the login page driver.get("Link of the captive portal") sleep(1) # we are sleeping here to let the site load first # Find and fill out the login form username_field = driver.find_element(By.NAME, "userId") password_field = driver.find_element(By.NAME, "password") login_button = driver.find_element(By.NAME, "Submit") # Enter your credentials username_field.send_keys("enter your username") password_field.send_keys("enter your password") # Click the login button login_button.click() # Optionally, wait for the next page to load and verify the login was successful # sleep(5) finally: # Close the browser driver.quit() Here I am using geckodriver, the webdriver for firefox but you can replace this with your choice of driver and browser. This script is for a basic captive portal with a username field, a password field and a Submit button. For your specific usecase you would need to replace "userid", "password" and "Submit" with the element names for that specific site. To do that, you can inspect the html code of site using F12 or right click and inspect. Then navigate to the form and get the field and button names. Great! Now we have the python script setup but it is still not done. We need to form a setup such that it runs automatically duing startup. Command Line Script - Below is the command line script for windows operating system, for other operating systems the script would be different. @echo off REM Change the path to the Python executable if necessary set PYTHON_PATH=C:\path\to\python.exe REM Change the path to the directory containing your Python script set SCRIPT_DIR=C:\path\to\your\script\directory REM Change to the script directory cd /d %SCRIPT_DIR% REM Run the Python script "%PYTHON_PATH%" selenium_script.py REM Pause to see any output pause Please adjust this script according to your operating system and file paths. Now we are ready with a command line script. What's next? We will the Task Scheduler inbuilt in Windows - Here’s the formatted list of steps to add a script to Task Scheduler to start at startup: Open Task Scheduler: Press Win + R to open the Run dialog. Type taskschd.msc and press Enter. This will open Task Scheduler. Create a Basic Task: In the Task Scheduler window, click on "Create Basic Task..." in the Actions pane on the right side. Name and Describe the Task: Name: Enter a name for your task, such as "Run Selenium Script at Startup." Description: Optionally, add a description for clarity. Choose a Trigger: Trigger: Select "When the computer starts" and click "Next." Start a Program: Action: Choose "Start a program" and click "Next." Select the Batch File: Program/Script: Click "Browse..." and navigate to your batch file (run_selenium_script.bat). Arguments (optional): Leave this blank. Start in (optional): You can specify the directory where your batch file is located if needed. Finish the Setup: Review the settings and click "Finish." Now when you restart your computer you will see that script runs automatically. This is one of the many applications of selenium, a powerful and widely-used open-source framework for automating web browsers. If you are interested in reading about it further, here is the link to the docs.

Activation Functions in Neural Networks

Prakriti Bhattacharya — Fri, 19 Jul 2024 06:43:58 GMT

While AGI may still be a few decades in the horizon, researchers have long been trying to mimic the human brain. Just like its namesake, the neural network works in the same way a neuron in the human body does. It receives stimuli, processes it and sends stimuli to the next neuron.This is achieved using layers- input, output and various hidden layers. Each layer has a varying number of units, depending on the amount of data to be processed and the nature of the data. Each layer has a specific activation function it uses to process the data. This function exists to ensure that processing is not restricted to learning linear relationships, which means that the network is capable of learning complex, non-linear patterns in data. It also regulates how signals propagate through the network, compress and normalise signals (e.g., sigmoid) or allow them to pass largely unchanged and mitigate issues like vanishing or exploding gradients.It is these activation functions which help us achieve something close to the firing patterns of neurons in biological brains. This makes the right selection of an activation function of a layer extremely important, contributing largely to learning speed, computational efficiency, and the network's ability to model complex relationships.The choice of activation function can mean the difference between a model that learns effectively and one that struggles with issues like vanishing gradients or slow convergence. As neural network architectures have evolved, so too has our understanding of activation functions and their impact on model performance. This evolution has led to the development of various types of activation functions, each with its own characteristics and strengths.Sigmoid:Function: f(x) = 1 / (1 + e^(-x))The sigmoid function compresses the input values to the range (0, 1). It makes the data smooth and continuously differentiable. Historically popular, it's now mainly used in the output layer for binary classification problems. The function saturates and kills gradients for very high or very low input values, which can slow down or halt learning in deep networks (vanishing gradient problem). Softmax:Function: f(x_i) = e^(x_i) / Σ(e^(x_j))Softmax is typically used in the output layer for multi-class classification problems. It converts a vector of real numbers into a probability distribution, where the probabilities sum to 1. This makes it easy to interpret the network's output as class probabilities. It's computationally more expensive than other activation functions, especially for a large number of classes. However, it is one of the more diverse activation functions typically used for probabilistic predictions, NLPs and Reinforcement Learning. Rectified Linear Unit (ReLU):Function: f(x) = max(0, x)ReLU is computationally efficient and helps alleviate the vanishing gradient problem. This is the activation function most commonly used for hidden layers. It allows for sparse activation in neural networks, as it sets all negative inputs to zero. This sparsity can be beneficial for computational efficiency and representation learning. However, ReLU can suffer from the "dying ReLU" problem, where neurons can get stuck in an inactive state if they consistently receive negative inputs. Leaky ReLU:Function: f(x) = x if x > 0, else αx (where α is a small constant, typically 0.01)Leaky ReLU addresses the dying ReLU problem by allowing a small gradient when the unit is not active. This helps prevent neurons from getting stuck in an inactive state. The small slope for negative inputs allows for some information to flow even when the neuron is not strongly activated, leading to more effective learning in deep networks. It doesn't completely discard negative inputs thus beneficial for tasks where negative signals are important. While still promoting some sparsity, it's less extreme than ReLU, it maintains most of ReLU's computational benefits and only slightly more complex than standard ReLU. Exponential Linear Unit (ELU) and SELU:Function: f(x) = x if x > 0, else α(e^x - 1) (where α is typically 1)ELU combines the benefits of ReLU (addressing vanishing gradients) with the ability to produce negative outputs unlike ReLU. The smooth function near zero helps in reducing the bias shift that ReLU can suffer from, helping in more robust learning and potentially better gradients. ELUs push mean activations closer to zero, which can help with internal covariate shift. ELU can result in faster learning and better generalization, but it's computationally more expensive than ReLU. Linear:Function: f(x) = xThe linear activation function is simply the identity function. The output is directly proportional to the input, which can be desirable in some cases. It needs no complex calculations, making it fast to compute and easy to use in conjunction with non-linear activations in other layers. It's typically used in the output layer for regression problems and isolate issues in network architecture or data preprocessing. While it doesn't add non-linearity to the network, it's useful when you want the network to output unbounded values. Swish:Function: f(x) = x * sigmoid(βx) (where β is a learnable parameter or set to 1)Swish is a relatively new activation function that has shown promising results in deep networks. It's smooth, non-monotonic, and unbounded above. The shape of the Swish function allows it to carry some small negative values, which can be beneficial for learning. Swish has a lower bound, which can help prevent exploding gradients in deep networks. It often outperforms ReLU in deep networks and also mitigate the bias shift problem that ReLU sometimes encounters but is computationally more expensive. Each function has its strengths and weaknesses. Sigmoid and tanh, while useful for specific tasks, can suffer from vanishing gradients. ReLU and its variants (Leaky ReLU, PReLU) have become popular due to their simplicity and effectiveness in mitigating gradient issues, though they may face problems like dying, inactive neurons. ELU and SELU aim to combine ReLU's benefits with improved handling of negative inputs, at the cost of increased computational complexity. Swish has shown promising results in deep networks but is also more computationally intensive than ReLU. The choice of activation function depends on the specific task, network architecture, and computational constraints. ReLU and its variants are often good default choices for hidden layers due to their simplicity and effectiveness. Sigmoid and softmax remain crucial for binary and multi-class classification output layers, respectively. For regression tasks, linear activation in the output layer is common. As research continues, new activation functions keep emerging, potentially offering improved performance or addressing specific challenges. The key is to understand the properties of each function and how they align with the requirements of your particular neural network application. Ultimately, the selection of activation functions often involves empirical testing and consideration of trade-offs between performance, computational efficiency, and the specific demands of the problem at hand. As with many aspects of deep learning, there's no one-size-fits-all solution, and the optimal choice may vary depending on the unique characteristics of each task and dataset.

Custom Object Detection using TensorFlow Lite

Rahul RG — Fri, 05 Jul 2024 03:58:16 GMT

TensorFlow Lite: Enhancing Mobile and Embedded Machine Learning TensorFlow Lite stands at the forefront of deploying machine learning models on mobile and embedded devices, embodying Google's cutting-edge adaptation of its broader TensorFlow framework. This lightweight, open-source solution enables machine learning functionalities to be efficiently executed on devices with limited computational power such as smartphones, tablets, and IoT devices. By compressing TensorFlow models to run faster and require less space, TensorFlow Lite ensures that these applications maintain high accuracy without compromising performance. It supports diverse machine learning tasks directly on devices, which is crucial for applications needing rapid responses and robust offline capabilities. Additionally, TensorFlow Lite provides an array of pre-built models and customization tools, empowering developers to integrate sophisticated AI capabilities seamlessly. This initiative dramatically enhances user experiences, offering intelligent, responsive interactions across various devices. Setting up the Environment Requirements Before starting, ensure you have the following: A Google account (for using Google Colab) Basic knowledge of Python and machine learning A dataset of images for object detection Reason for using Google Colab Google Colab offers a free environment with a GPU, perfect for training deep learning models. You get a GPU with 15 GB of RAM for free, which is usually expensive. This is great for training TensorFlow models, especially if you don't need to train many models often. To start, open Google Colab and create a new notebook. Gathering and Labeling Training Data Gather a dataset of images that include the objects you want to detect. Aim for at least 1000 images with varied backgrounds and lighting conditions to have higher accuracy of prediction. Labeling Images Using LabelImg Use LabelImg to annotate the images. Each image should have an XML file containing the annotations in the PascalVOC format. Save these files in a folder named 'images'. Installing TensorFlow Object Detection API First, we'll install the TensorFlow Object Detection API in this Google Colab instance. This requires cloning the TensorFlow models repository and running a couple installation commands. # Clone the tensorflow models repository from GitHub !pip uninstall Cython -y !git clone --depth 1 https://github.com/tensorflow/models Then execute the following commands import re with open('/content/models/research/object_detection/packages/tf2/setup.py') as f: s = f.read() with open('/content/models/research/setup.py', 'w') as f: # Set fine_tune_checkpoint path s = re.sub('tf-models-official>=2.5.1', 'tf-models-official==2.8.0', s) f.write(s) Configuring GPU for training !pip install pyyaml==5.3 !pip install /content/models/research/ !pip install tensorflow==2.8.0 !pip install tensorflow_io==0.23.1 !wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin !mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 !wget http://developer.download.nvidia.com/compute/cuda/11.0.2/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb !dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb !apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub !apt-get update && sudo apt-get install cuda-toolkit-11-0 !export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH !python /content/models/research/object_detection/builders/model_builder_tf2_test.py Expected Output Running tests under Python 3.10.12: /usr/bin/python3 [ RUN ] ModelBuilderTF2Test.test_create_center_net_deepmac 2024-03-22 23:48:56.660438: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. W0322 23:48:57.221283 134667806457856 model_builder.py:1112] Building experimental DeepMAC meta-arch. Some features may be omitted. INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_deepmac): 3.59s I0322 23:48:57.940035 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_deepmac): 3.59s [ OK ] ModelBuilderTF2Test.test_create_center_net_deepmac [ RUN ] ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True) INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)): 1.16s I0322 23:48:59.095793 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)): 1.16s [ OK ] ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True) [ RUN ] ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False) INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)): 0.49s I0322 23:48:59.584455 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)): 0.49s [ OK ] ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False) [ RUN ] ModelBuilderTF2Test.test_create_center_net_model_from_keypoints INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model_from_keypoints): 0.5s I0322 23:49:00.089346 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model_from_keypoints): 0.5s [ OK ] ModelBuilderTF2Test.test_create_center_net_model_from_keypoints [ RUN ] ModelBuilderTF2Test.test_create_center_net_model_mobilenet INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model_mobilenet): 4.25s I0322 23:49:04.342767 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model_mobilenet): 4.25s [ OK ] ModelBuilderTF2Test.test_create_center_net_model_mobilenet [ RUN ] ModelBuilderTF2Test.test_create_experimental_model INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_experimental_model): 0.0s I0322 23:49:04.344451 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_experimental_model): 0.0s . . more output . . [ OK ] ModelBuilderTF2Test.test_unknown_meta_architecture [ RUN ] ModelBuilderTF2Test.test_unknown_ssd_feature_extractor INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_unknown_ssd_feature_extractor): 0.0s I0322 23:49:30.352874 134667806457856 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_unknown_ssd_feature_extractor): 0.0s [ OK ] ModelBuilderTF2Test.test_unknown_ssd_feature_extractor ---------------------------------------------------------------------- Ran 24 tests in 36.002s OK (skipped=1) Gather and Label Images To prepare your system for image annotation, it is essential to first install Anaconda Prompt. Anaconda serves as a robust tool that simplifies package management and deployment, making it ideal for establishing environments for various tasks. Upon installing Anaconda, you can effortlessly install labelImg, a graphical image annotation tool. LabelImg enables efficient labeling of objects within images, which is crucial for training machine learning models. After installing labelImg via Anaconda, you can commence annotating your images by drawing bounding boxes around the objects of interest directly within the tool. This setup will streamline your preparation process for any image-based machine learning projects.Let's get started with it. Download Anaconda Anaconda is a software system that helps manage and organize tools for data science and machine learning. It makes it easier to install and handle different programs and environments needed for scientific projects. Anaconda supports popular languages like Python and R, making it valuable for research and development tasks. To download Anaconda, visit the Anaconda website, go to the Downloads section, and choose the installer for your operating system (Windows, macOS, Linux). Download the file, run the installer, follow the prompts to agree to the license and select the installation location if needed. Once installed, verify by opening Anaconda Navigator or Anaconda Prompt. Open the Anaconda Prompt Terminal and it should look like LabelIMG (base) C:\Users\User1>pip install labelimg After Labelimg is downloaded and installed type the following code to bootup the application up. (base) C:\Users\User1>labelimg This should open up the Labelimg application Now Open up a directory with images to start annotating by making rectangles over the different objects to be detected with their respective labels.These labels are also known as class.Make sure the format of annotation is same throught all the images.PascalVOC format is the default format and is recommended to keep the same.After making the annotations for the respective labels, it should look look like this. After annotating your images with LabelImg, organize all images and XML files into a single folder. Create a ZIP file by right-clicking the folder and selecting the compress option. Then, open Google Drive, click "+ New," select "File upload," and choose the ZIP file to upload. Now, type the following command to link your google drive to the Google Colab project and access files on the drive in the project. from google.colab import drive drive.mount('/content/gdrive') !cp /content/gdrive/Your_Location/images.zip /content #change this to the location of images.zip on your drive To prepare your dataset for machine learning, you need to create three folders: train, validation, and test. Begin by allocating 85% of your images to the train folder, ensuring your model has a substantial amount of data to learn from. Next, place 10% of the images in the validation folder to fine-tune the model's hyperparameters and prevent overfitting. Finally, allocate the remaining 5% of the images to the test folder, which will be used to evaluate the model's performance on unseen data. This division helps in creating a robust and reliable machine learning model. I have provided the python script to do the same. Feel free to change the values of allocation to suite your dataset. from pathlib import Path import random import os import sys all_images_path = '/content/images/all' train_images_path = '/content/images/train' validation_images_path = '/content/images/validation' test_images_path = '/content/images/test' jpeg_files = [path for path in Path(all_images_path).rglob('*.jpeg')] jpg_files = [path for path in Path(all_images_path).rglob('*.jpg')] png_files = [path for path in Path(all_images_path).rglob('*.png')] bmp_files = [path for path in Path(all_images_path).rglob('*.bmp')] if sys.platform == 'linux': JPEG_files = [path for path in Path(all_images_path).rglob('*.JPEG')] JPG_files = [path for path in Path(all_images_path).rglob('*.JPG')] all_files = jpg_files + JPG_files + png_files + bmp_files + JPEG_files + jpeg_files else: all_files = jpg_files + png_files + bmp_files + jpeg_files total_files = len(all_files) print('Total images: %d' % total_files) train_split = 0.85 # 85% of the files go to train validation_split = 0.1 # 10% go to validation test_split = 0.05 # 5% go to test train_count = int(total_files * train_split) validation_count = int(total_files * validation_split) test_count = total_files - train_count - validation_count print('Images moving to train: %d' % train_count) print('Images moving to validation: %d' % validation_count) print('Images moving to test: %d' % test_count) for _ in range(train_count): selected_file = random.choice(all_files) file_name = selected_file.name file_stem = selected_file.stem parent_directory = selected_file.parent xml_file_name = file_stem + '.xml' os.rename(selected_file, train_images_path + '/' + file_name) os.rename(os.path.join(parent_directory, xml_file_name), os.path.join(train_images_path, xml_file_name)) all_files.remove(selected_file) for _ in range(validation_count): selected_file = random.choice(all_files) file_name = selected_file.name file_stem = selected_file.stem parent_directory = selected_file.parent xml_file_name = file_stem + '.xml' os.rename(selected_file, validation_images_path + '/' + file_name) os.rename(os.path.join(parent_directory, xml_file_name), os.path.join(validation_images_path, xml_file_name)) all_files.remove(selected_file) for _ in range(test_count): selected_file = random.choice(all_files) file_name = selected_file.name file_stem = selected_file.stem parent_directory = selected_file.parent xml_file_name = file_stem + '.xml' os.rename(selected_file, test_images_path + '/' + file_name) os.rename(os.path.join(parent_directory, xml_file_name), os.path.join(test_images_path, xml_file_name)) all_files.remove(selected_file) LabelMap and TFrecord TFRecord is a format used with TensorFlow to store data efficiently. It's great for large datasets, such as images or sequences, because it saves the data as binary records, making loading and processing faster. This helps improve the performance and scalability of TensorFlow models. LabelMaps are files that match label numbers to readable names. They're crucial in TensorFlow, especially for tasks like object detection where the model outputs numeric class IDs. The LabelMap helps translate these numbers back into understandable labels, making it easier to interpret and evaluate what the model is doing. Together, TFRecords and LabelMaps make managing and understanding data smoother, aiding in building strong machine learning models with TensorFlow. Now to declare the classes used during the annotation into this code. %%bash cat <> /content/labelmap.txt Class 1 Class 2 EOF Now download and run the following scripts of my github repository # Download data conversion scripts ! wget https://raw.githubusercontent.com/Rahul-2004/ObjectDetectionTFlite/main/create_csv.py ! wget https://raw.githubusercontent.com/Rahul-2004/ObjectDetectionTFlite/main/create_tfrecord.py !python3 create_csv.py Reviewing Data Integrity Before proceeding with training the model, it is crucial to verify that each image has a corresponding XML file. This step is important because, in Google Colab, the created TFRecord will be stored in memory for a certain period. If we create a TFRecord and later discover that some images are missing their XML files, we won't be able to rerun the model until Colab deletes the existing TFRecord from memory. This can cause delays and make troubleshooting more difficult. Ensuring all necessary files are present now will streamline the process and help avoid complications during model training.The following code ensures the same. import os import pandas as pd # Define paths train_csv_path = '/content/images/train_labels.csv' validation_csv_path = '/content/images/validation_labels.csv' train_folder = '/content/images/train' validation_folder = '/content/images/validation' # Load CSV files train_df = pd.read_csv(train_csv_path) validation_df = pd.read_csv(validation_csv_path) # Check for missing PNG files in train folder missing_png_train = [] for index, row in train_df.iterrows(): filename = row['filename'] png_path = os.path.join(train_folder, filename.replace('.xml', '.png')) if not os.path.exists(png_path): missing_png_train.append(filename) # Check for missing PNG files in validation folder missing_png_validation = [] for index, row in validation_df.iterrows(): filename = row['filename'] png_path = os.path.join(validation_folder, filename.replace('.xml', '.png')) if not os.path.exists(png_path): missing_png_validation.append(filename) # Check for missing XML files in train folder missing_xml_train = [] for filename in os.listdir(train_folder): if filename.endswith('.xml') and not os.path.exists(os.path.join(train_folder, filename.replace('.xml', '.png'))): missing_xml_train.append(filename) # Check for missing XML files in validation folder missing_xml_validation = [] for filename in os.listdir(validation_folder): if filename.endswith('.xml') and not os.path.exists(os.path.join(validation_folder, filename.replace('.xml', '.png'))): missing_xml_validation.append(filename) # Delete missing names from train CSV train_df = train_df[~train_df['filename'].isin(missing_png_train + missing_xml_train)] train_df.to_csv(train_csv_path, index=False) # Delete missing names from validation CSV validation_df = validation_df[~validation_df['filename'].isin(missing_png_validation + missing_xml_validation)] validation_df.to_csv(validation_csv_path, index=False) # Print results if missing_png_train: print(f'Missing PNG files in train folder: {missing_png_train}') if missing_png_validation: print(f'Missing PNG files in validation folder: {missing_png_validation}') if missing_xml_train: print(f'Missing XML files in train folder: {missing_xml_train}') if missing_xml_validation: print(f'Missing XML files in validation folder: {missing_xml_validation}') Once we are sure about all the files we can head toward creating an error free tfrecord. !python3 create_tfrecord.py --csv_input=images/train_labels.csv --labelmap=labelmap.txt --image_dir=images/train --output_path=train.tfrecord !python3 create_tfrecord.py --csv_input=images/validation_labels.csv --labelmap=labelmap.txt --image_dir=images/validation --output_path=val.tfrecord train_record_fname = '/content/train.tfrecord' val_record_fname = '/content/val.tfrecord' label_map_pbtxt_fname = '/content/labelmap.pbtxt' Configure Training We are now at the stage of configuring the model training process. To begin, you can visit the TensorFlow Model Zoo, which offers a comprehensive repository of pre-trained models designed for various tasks, including object detection. Explore the available models at ModelZoo. Additionally, for insights into TensorFlow Lite object detection models and their performance comparison, you can refer to the detailed analysis provided by Evan Juras, click here to visit the documentation. This resource compares the effectiveness of various models based on different metrics, helping you choose the most suitable one for your project requirements. Once you have selected the optimal model, proceed by specifying its name in the chosen_model variable to proceed with configuring and training your model accordingly. chosen_model = "ssd-mobilenet-v2-fpnlite-320" MODELS_CONFIG = { "ssd-mobilenet-v2": { "model_name": "ssd_mobilenet_v2_320x320_coco17_tpu-8", "base_pipeline_file": "ssd_mobilenet_v2_320x320_coco17_tpu-8.config", "pretrained_checkpoint": "ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz", }, "efficientdet-d0": { "model_name": "efficientdet_d0_coco17_tpu-32", "base_pipeline_file": "ssd_efficientdet_d0_512x512_coco17_tpu-8.config", "pretrained_checkpoint": "efficientdet_d0_coco17_tpu-32.tar.gz", }, "ssd-mobilenet-v2-fpnlite-320": { "model_name": "ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8", "base_pipeline_file": "ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config", "pretrained_checkpoint": "ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz", }, } model_name = MODELS_CONFIG[chosen_model]["model_name"] pretrained_checkpoint = MODELS_CONFIG[chosen_model]["pretrained_checkpoint"] base_pipeline_file = MODELS_CONFIG[chosen_model]["base_pipeline_file"] Using Pre-trained Models First, make a new folder named "mymodel" under "/content/models/". This folder will keep all the important pre-trained weights. These weights have learned information from training on a big dataset, especially for tasks like object detection. Once the folder is ready, go into it using %cd /content/models/mymodel/. Next, get the pre-trained model weights from TensorFlow's website. These weights come in a compressed tarfile format. Extract this file into the current folder to get the pre-trained weights needed to start your custom model. Also, get the training setup file for the model from TensorFlow's GitHub page. This file has important settings and details needed to set up and train your custom object detection model. %mkdir /content/models/mymodel/ %cd /content/models/mymodel/ import tarfile download_tar = 'http://download.tensorflow.org/models/object_detection/tf2/20200711/' + pretrained_checkpoint !wget {download_tar} tar = tarfile.open(pretrained_checkpoint) tar.extractall() tar.close() download_config = 'https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/configs/tf2/' + base_pipeline_file !wget {download_config} Setting Custom Parameters When setting up training for your object detection model, you can adjust several important settings to fit your project needs. The num_steps parameter, set to 10,000 by default, controls how many training steps the model will take. You can change this number to make the training longer or shorter, depending on your goals.Usually more the steps more the model learns to make accurate predictions. The batch_size parameter changes based on the model you choose. For the 'efficientdet-d0' model, the batch size is 8, meaning 8 images are processed at once during each training step. This setting affects how fast the training runs and how much memory it uses. If your GPU or training setup needs a different batch size, you can change this number. For other models not specifically mentioned, the default batch size is 24. By adjusting these settings—num_steps, chosen_model, and batch_size—you can make the training process more efficient and better suited to your project's needs. This helps your model learn to detect objects more accurately for your application. num_steps = 10000 if chosen_model == 'efficientdet-d0': batch_size = 8 else: batch_size = 24 pipeline_fname = "/content/models/mymodel/" + base_pipeline_file fine_tune_checkpoint = "/content/models/mymodel/" + model_name + "/checkpoint/ckpt-0" def get_num_classes(pbtxt_fname): from object_detection.utils import label_map_util label_map = label_map_util.load_labelmap(pbtxt_fname) categories = label_map_util.convert_label_map_to_categories( label_map, max_num_classes=90, use_display_name=True ) category_index = label_map_util.create_category_index(categories) return len(category_index.keys()) num_classes = get_num_classes(label_map_pbtxt_fname) print("Total classes:", num_classes) Then we need to tweak a few parameters in the pipline_file.config file to run the model smoothly. import re %cd /content/models/mymodel print('writing custom configuration file') with open(pipeline_fname) as f: s = f.read() with open('pipeline_file.config', 'w') as f: # Set fine_tune_checkpoint path s = re.sub('fine_tune_checkpoint: ".*?"', 'fine_tune_checkpoint: "{}"'.format(fine_tune_checkpoint), s) # Set tfrecord files for train and test datasets s = re.sub( '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/train)(.*?")', 'input_path: "{}"'.format(train_record_fname), s) s = re.sub( '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/val)(.*?")', 'input_path: "{}"'.format(val_record_fname), s) # Set label_map_path s = re.sub( 'label_map_path: ".*?"', 'label_map_path: "{}"'.format(label_map_pbtxt_fname), s) # Set batch_size s = re.sub('batch_size: [0-9]+', 'batch_size: {}'.format(batch_size), s) # Set training steps, num_steps s = re.sub('num_steps: [0-9]+', 'num_steps: {}'.format(num_steps), s) # Set number of classes num_classes s = re.sub('num_classes: [0-9]+', 'num_classes: {}'.format(num_classes), s) # Change fine-tune checkpoint type from "classification" to "detection" s = re.sub( 'fine_tune_checkpoint_type: "classification"', 'fine_tune_checkpoint_type: "{}"'.format('detection'), s) f.write(s) Saving the checkpoints in their respective directories. pipeline_file = '/content/models/mymodel/pipeline_file.config' model_dir = '/content/training/' Training the Model The model is ready for training. Now, we will start TensorBoard to monitor the training process. TensorBoard is a helpful tool that shows us how the training is going by displaying graphs of important metrics like loss and accuracy. It also provides visualizations of the model's structure and the data being used. By watching TensorBoard, we can make sure the training is on track and make adjustments if needed to improve the model's performance. %load_ext tensorboard %tensorboard --logdir '/content/training/train' To start training, !python /content/models/research/object_detection/model_main_tf2.py \ --pipeline_config_path={pipeline_file} \ --model_dir={model_dir} \ --alsologtostderr \ --num_train_steps={num_steps} \ --sample_1_of_n_eval_examples=1 Once the training is done successfully, we will conver the model with the learned weights from the training data, Let us export it to be able to use it on any device. Exporting the model as TFlite First, we need to export the model graph (a file that contains information about the architecture and weights) to a TensorFlow Lite-compatible format. We'll do this using the export_tflite_graph_tf2.py script. !mkdir /content/custom_model_lite output_directory = '/content/custom_model_lite' # Path to training directory (the conversion script automatically chooses the highest checkpoint file) last_model_path = '/content/training' !python /content/models/research/object_detection/export_tflite_graph_tf2.py \ --trained_checkpoint_dir {last_model_path} \ --output_directory {output_directory} \ --pipeline_config_path {pipeline_file} import tensorflow as tf converter = tf.lite.TFLiteConverter.from_saved_model('/content/custom_model_lite/saved_model') tflite_model = converter.convert() with open('/content/custom_model_lite/detect.tflite', 'wb') as f: f.write(tflite_model) Testing the Model Once the model is trained and downloaded successfully, make a python file with the following code to run the model. import os import argparse import cv2 import numpy as np import sys import glob import importlib.util # Argument parser for command line arguments parser = argparse.ArgumentParser() parser.add_argument('--modeldir', help='Folder the .tflite file is located in', required=True) parser.add_argument('--graph', help='Name of the .tflite file, if different than detect.tflite', default='detect.tflite') parser.add_argument('--labels', help='Name of the labelmap file, if different than labelmap.txt', default='labelmap.txt') parser.add_argument('--threshold', help='Minimum confidence threshold for displaying detected objects', default=0.5) parser.add_argument('--image', help='Name of the single image to perform detection on. To run detection on multiple images, use --imagedir', default=None) parser.add_argument('--imagedir', help='Name of the folder containing images to perform detection on. Folder must contain only images.', default=None) parser.add_argument('--save_results', help='Save labeled images and annotation data to a results folder', action='store_true') parser.add_argument('--noshow_results', help='Don\'t show result images (only use this if --save_results is enabled)', action='store_false') args = parser.parse_args() # Define the paths and filenames MODEL_NAME = args.modeldir GRAPH_NAME = args.graph LABELMAP_NAME = args.labels min_conf_threshold = float(args.threshold) save_results = args.save_results show_results = args.noshow_results IM_NAME = args.image IM_DIR = args.imagedir # Validate image and imagedir arguments if (IM_NAME and IM_DIR): print('Error! Please only use the --image argument or the --imagedir argument, not both. Issue "python TFLite_detection_image.py -h" for help.') sys.exit() if (not IM_NAME and not IM_DIR): IM_NAME = 'test1.jpg' # Import the TensorFlow Lite interpreter pkg = importlib.util.find_spec('tflite_runtime') if pkg: from tflite_runtime.interpreter import Interpreter else: from tensorflow.lite.python.interpreter import Interpreter # Get the current working directory CWD_PATH = os.getcwd() # Define the image paths if IM_DIR: PATH_TO_IMAGES = os.path.join(CWD_PATH, IM_DIR) images = glob.glob(PATH_TO_IMAGES + '/*.jpg') + glob.glob(PATH_TO_IMAGES + '/*.png') + glob.glob(PATH_TO_IMAGES + '/*.bmp') if save_results: RESULTS_DIR = IM_DIR + '_results' elif IM_NAME: PATH_TO_IMAGES = os.path.join(CWD_PATH, IM_NAME) images = glob.glob(PATH_TO_IMAGES) if save_results: RESULTS_DIR = 'results' if save_results: RESULTS_PATH = os.path.join(CWD_PATH, RESULTS_DIR) if not os.path.exists(RESULTS_PATH): os.makedirs(RESULTS_PATH) # Load the TensorFlow Lite model and labels PATH_TO_CKPT = os.path.join(CWD_PATH, MODEL_NAME, GRAPH_NAME) PATH_TO_LABELS = os.path.join(CWD_PATH, MODEL_NAME, LABELMAP_NAME) with open(PATH_TO_LABELS, 'r') as f: labels = [line.strip() for line in f.readlines()] if labels[0] == '???': del(labels[0]) interpreter = Interpreter(model_path=PATH_TO_CKPT) interpreter.allocate_tensors() # Get model details input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() height = input_details[0]['shape'][1] width = input_details[0]['shape'][2] floating_model = (input_details[0]['dtype'] == np.float32) input_mean = 127.5 input_std = 127.5 outname = output_details[0]['name'] if ('StatefulPartitionedCall' in outname): boxes_idx, classes_idx, scores_idx = 1, 3, 0 else: boxes_idx, classes_idx, scores_idx = 0, 1, 2 # Process each image for image_path in images: image = cv2.imread(image_path) image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) imH, imW, _ = image.shape image_resized = cv2.resize(image_rgb, (width, height)) input_data = np.expand_dims(image_resized, axis=0) if floating_model: input_data = (np.float32(input_data) - input_mean) / input_std interpreter.set_tensor(input_details[0]['index'], input_data) interpreter.invoke() boxes = interpreter.get_tensor(output_details[boxes_idx]['index'])[0] classes = interpreter.get_tensor(output_details[classes_idx]['index'])[0] scores = interpreter.get_tensor(output_details[scores_idx]['index'])[0] detections = [] for i in range(len(scores)): if ((scores[i] > min_conf_threshold) and (scores[i] <= 1.0)): ymin = int(max(1, (boxes[i][0] * imH))) xmin = int(max(1, (boxes[i][1] * imW))) ymax = int(min(imH, (boxes[i][2] * imH))) xmax = int(min(imW, (boxes[i][3] * imW))) cv2.rectangle(image, (xmin, ymin), (xmax, ymax), (10, 255, 0), 2) object_name = labels[int(classes[i])] label = '%s: %d%%' % (object_name, int(scores[i] * 100)) labelSize, baseLine = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.7, 1) label_ymin = max(ymin, labelSize[1] + 10) cv2.rectangle(image, (xmin, label_ymin - labelSize[1] - 10), (xmin + labelSize[0], label_ymin + baseLine - 10), (255, 255, 255), cv2.FILLED) cv2.putText(image, label, (xmin, label_ymin - 7), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2) detections.append([object_name, scores[i], xmin, ymin, xmax, ymax]) if show_results: cv2.imshow('Object detector', image) if cv2.waitKey(0) == ord('q'): break if save_results: image_fn = os.path.basename(image_path) image_savepath = os.path.join(CWD_PATH, RESULTS_DIR, image_fn) base_fn, ext = os.path.splitext(image_fn) txt_result_fn = base_fn + '.txt' txt_savepath = os.path.join(CWD_PATH, RESULTS_DIR, txt_result_fn) cv2.imwrite(image_savepath, image) with open(txt_savepath, 'w') as f: for detection in detections: f.write('%s %.4f %d %d %d %d\n' % (detection[0], detection[1], detection[2], detection[3], detection[4], detection[5])) cv2.destroyAllWindows() This code takes in various arguments to run the prediction. For running the model on a directory of images, python script_name.py --modeldir "my_model_directory" --imagedir "my_image_directory" For running the model on a single image, python script_name.py --modeldir "my_model_directory" --image "my_image.jpg" Replace script_name.py with the name of the script ,my_model_directory with the path of the folder with all the model files and my_image_directory and my_image.jpg with their respective paths.Remember to mention the path within quotes.

Automating MySQL Backups

Aryan — Wed, 26 Jun 2024 13:15:32 GMT

Backing up a MySQL database running on production is crucial to ensure data integrity and availability. In the event of hardware failures, software bugs, human errors, or malicious attacks, a backup provides a reliable way to restore lost or corrupted data, minimizing downtime and operational disruption. Regular backups also support data recovery during unexpected incidents and protecting valuable information. By implementing a consistent backup strategy, we can safeguard their critical data, maintain user trust, and comply with regulatory requirements, ultimately contributing to a robust and resilient system. A simple backup of a MySQL database can be taken manually by simply exporting the database from the MySQL Workbench, however we would not be exploring that in this article. A MySQL database is backed up by mysqldump.exe which is located in MySQL\bin\. mysqldump.exe creates a logical backup by exporting the database structure and data into a SQL script file that can be used to recreate the database. This can be done manually using a gui where you can select which databases and which tables to backup or we can write a script to do that. Create backupscript.bat and paste the following code in it. @echo off cd "path\to\mysql\bin" for /f "tokens=2 delims= " %%a in ('date /t') do set mydate=%%a set mydate=%mydate:/=-% set mydate=%mydate: =% set mydate=%mydate:~10,4%%mydate:~4,2%%mydate:~7,2% set backup_path=path/to/backup_folder set backup_name=_%mydate% mysqldump --defaults-extra-file=path\to\my.cnf --all-databases --routines --events --result-file="%backup_path%\%backup_name%.sql" if %ERRORLEVEL% neq 0 ( echo [%date%] Backup failed: error during dump creation >> "%backup_path%\mysql_backup_log.txt" ) else ( echo [%date%] Backup successful >> "%backup_path%\mysql_backup_log.txt" ) NOTE: Make sure to replace all the paths and the database names. Now create my.cnf and add the following to it. [client] user=username password=password Add your username and password. This is a config file which we will use to fetch details of the user to run the mysqldump.exe command. It is not recommended to enter sensitive information like passwords in the terminal hence this approach is better. Now we can schedule to run this script using the task scheduler. Open the tasak scheduler and click on Task Scheduler Library in the left panel. Then in the right panel select Create Basic Task... Enter the basic information and select on run script and select the backupscript.bat. Enter your desired interval for backups and we're done! Now your database will be backed up and you will also have a log of all the backups. If ever you accidentally lose your data you would regret not doing this. So if you're running any projects in production make sure you backup your database. It takes less than 5 minutes and you'd be better safe than sorry.

Serving Node Applications on Microsoft IIS

Aryan — Sun, 23 Jun 2024 18:38:24 GMT

Over the weekend I was bored and tried hosting node applications on IIS on a server. Since currently I have only tried wordpress I had no idea how to go about this which might be why you are here, so lets dive straight into how to get it running. So first of all if you haven't done this before you will probably not have node installed on your server. So first lets install everything that we will need. Prerequisites If you do not have IIS (Internet Information Services) enabled; you will have to enable it. Search Turn Windows Features On or Off and check the box next to Internet Information Services. Expand the Internet Information Services node to select additional features like Web Management Tools and World Wide Web Services. Ensure at least the following are selected: Web Management Tools: Includes IIS Management Console. World Wide Web Services: Includes various services for hosting websites and applications. Install node from here. Make sure that you add the path to node.exe in your environment variables. In most cases it would be C:\Program Files\nodejs\. Install iisnode from here. Install URL Rewrite from here. Creating and Setting up the Site Open IIS Manager and click on your device in the connections panel on the left to expand it. Right click Sites and click on the Add Website... option to create a new site, and add the domain name you want it to show up on in the hostname and check the start website immediately. Add 127.0.0.1 to your Site from the bindings in the right pane in your site's page in IIS. Check if iisnode is installed in modules. Locate the folder C:\intepub\wwwroot\ and create a folder with the same name as your site. Add the following files to C:\inetpup\wwwroot\{site name}: server.js web.config Open the terminal and install ExpressJs using npm install express. Write your server script in server.js to create an express server. var express = require("express"); var app = express(); app.get("/ping", function(req, res) { res.send("Pong!"); }); app.listen(process.env.PORT, () => { console.log("listening"); }); Add the following to web.config This web.config file is written in XML and is used to configure settings for web applications hosted on Internet Information Services (IIS). It allows you to define how IIS should handle requests, manage security settings, rewrite URLs, handle custom error messages, and integrate with specific modules like iisnode for hosting Node.js applications. Note: If you are having trouble creating or editing files in C:\inetput\wwwroot\{your site} , right click and open properties. Add your user if it does not exist and check the full control box to make the process of editing and creating files easier. In most cases this requires administrative control. And with that we have served node applications on IIS!

A Hacky Way to Scrape Linkedin

Aryan — Sun, 02 Jun 2024 11:21:01 GMT

Recently, I was tasked with building a Chrome extension that scrapes LinkedIn profiles to gather important information such as name, bio, location, follower count, and connection count from a list of predetermined URLs. Once collected, this data needs to be saved to a database using a POST request with Sequelize as the ORM. This is fairly simple and straightforward to do. Set up a SQL database and create a User model using Sequalize. Create an Express server. Develop the Chrome Extension. I was expecting this to be very simple and thought it wouldn't take more than 30-40 minutes. I developed the backend for this as you would do with any other web dev project and this was probably the easiest task. Developing the chrome extension is what took the longest. I'll walk you through my thought process and what mistakes I made while developing this. I will focus more on the scraping part and the Chrome Extension in this article and not the creation of the REST API using Node and Express. You can read more about how to create REST APIs using Node.js and Express here. Project Structure . ├── Backend | ├── Models | | └── User.js | ├── node_modules | ├── .env | ├── .gitignore | ├── db.js | ├── server.js | ├── package.json | └── package-lock.json └── Extension ├── manifest.json ├── index.html ├── icon.png ├── style.css ├── script.js ├── content.js └── background.js Models/User.js import { DataTypes } from "sequelize"; import sequelize from "../db.js"; const User = sequelize.define('User', { id: { type: DataTypes.INTEGER, primaryKey: true, autoIncrement: true, }, name: { type: DataTypes.STRING, allowNull: false, }, url: { type: DataTypes.STRING, allowNull: false, }, about: { type: DataTypes.STRING, allowNull: true, }, bio: { type: DataTypes.STRING, allowNull: true, }, location: { type: DataTypes.STRING, allowNull: true, }, followerCount: { type: DataTypes.NUMBER, allowNull: false, }, connectionCount: { type: DataTypes.NUMBER, allowNull: true, }, }, { tableName: 'users', timestamps: false, }); export default User; ./Db.js import dotenv from 'dotenv'; import { Sequelize } from "sequelize"; dotenv.config(); const sequelize = new Sequelize(process.env.DB, process.env.USER, process.env.PASSWORD, { host: 'localhost', dialect: 'mysql', }) async function testConnection() { try { await sequelize.authenticate(); console.log('Connection has been established successfully.'); } catch (error) { console.error('Unable to connect to the database:', error); } } testConnection(); export default sequelize; ./server.js import cors from 'cors'; import dotenv from 'dotenv'; import express from 'express'; import User from './models/User.js'; dotenv.config(); const app = express(); app.use(express.json()); app.use(cors()); const port = process.env.PORT || 3000; app.post('/getinfo', async (req, res) => { try { const { name, url, about, bio, location, followerCount, connectionCount } = req.body; const newUser = await User.create({ name: name, url: url, about: about, bio: bio, location: location, followerCount: followerCount, connectionCount:connectionCount, }) res.status(200).send(newUser); } catch (err) { console.log(err.message); } }) app.get('/ping', (req, res) => { res.send('pong'); }) app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Next I had to figure out how to do basic stuff using Chrome Extensions. Since I had not much experience in this to begin with I spent some time learning this. So as you know that we can have content_scripts in our extensions which will inject javascript scripts in the client's browser, which will run on all websites which match with the listed matches in your manifest.json. So let us break down the task of creating this Chrome Extension into smaller chunks: Get list of linkedin profile urls from a popup. Open each url from the list of urls. Get user info from each linkedin page. Perfom a POST request to our backend server to save it to the database. The first chunk is actually very easy. Create a manifest.json and add the following to it. { "manifest_version": 3, "name": "Linkedin Data Extractor", "description": "A chrome extension for a TechKnowHow article.", "version": "1.0", "action": { "default_popup": "index.html", "default_icon": "icon.png" }, "permissions": [ "activeTab", "scripting" ], "content_scripts": [ { "matches": ["https://www.linkedin.com/in/*"], "js": ["content.js"], "run_at": "document_idle" } ], "background": { "service_worker": "background.js" } } Note the permissions, content_scripts, and background scrips. Now create index.html and style.css and append the follwing to it. LinkedIn Data Extractor

Extract LinkedIn User Data

body{ width: 300px; display: flex; flex-direction: column; gap: 10px; align-items: center; } textarea{ width: 95%; } button{ height: 40px; width: 100px; background-color: rgb(149, 205, 65); color: black; border: 0.5px solid black; } Now let us write the logic for getting the linikedin urls and openinng them in individual tabs one by one in script.js. document.getElementById('extractButton').addEventListener('click', async () => { const urls = document.getElementById('linkedinUrls').value.split('\n').filter(url => url.trim()); if (urls.length < 3) { alert('Please provide at least 3 LinkedIn profile URLs'); return; } for (const url of urls) { await chrome.tabs.create({ url }); } }); okay so what is exactly happening here? We've added an event listener to the button and we are getting the urls and if they're less than 3 then we are throwing an alert saying Please provide at least 3 LinkedIn profile URLs. And if we get more than 3 urls then we are opening them one by one in new tabs using chrome.tabs.create(). While creating Chrome Extensions to perform any task which deals with getting stuff from the webpage or opening new tabs, etc we need to use the chrome API which enables us to perform all these operations. You can find more stuff on Chrome APIs here. With that we are able to open the urls one by one now we just need to get the info from the pages. Okay so my first instinct was to just document.querySelector(classname)and get the details. So I tried to do that for just the name at first to see if it works. And it worked! So I continued with the same approach to get the details of all the fields. Now here's when I encountered a problem. I was able to get name, location, bio but null was being returned for all the remaining stuff, and I just couldn't figure out why. If I was getting null while retreiving all the fields I would've realized what the problem was much earlier but since I was facing this issue only for half the things it took me a considerable amount of time to understand where I was going wrong. My debugging process So since I was running a content_script it was being run in the client's browser so I logged everything into the user's console like so. // content.js const nameElement = document.querySelector("h1").innerText; const bioElement = document.querySelector(".text-body-medium").innerText; const locElement = document.querySelector(".text-body-small.inline.t-black--light.break-words").innerText; const followersElement = document.querySelector('.artdeco-card .rQrgCqdAxxLhIAcLhxdsifdagjxISOpE span').innerText; const aboutElement = document.querySelector(".JSzNEGEyfojpwDGomLCFeVXPtVfgJfKE span").innerText; const connectionElement = document.querySelector(".OnsbwwsPVDGkAkHfUohWiCwsWEWrcqkY").innerText; console.log(nameElement); console.log(bioElement); console.log(locElement); console.log(followersElement); console.log(aboutElement); console.log(connectionElement); Only name, bio & locaion were logged and for everything else I got the following error. Cannot get innerText for null, which basically means that the html tag that we were trying to locate was not found. What was the problem? Whenever we reload the page takes some time to load. If we try to access these html tags before they are even generated we will be returned null. So why were we able to get name, bio & location? This was probably because the class names for name, bio & location do not change, on the contrary the class names for the remaining tags seem to be random and probably generated instead of defined. These classnames might be different at the time you read this but the class names for name, bio & location would still be same. How do we fix this? Since the deadline for this task was right around the corner I came up with a hacky solution. I tried to get the tag every 100 miliseconds for 10 seconds. If I did not get the tag after 10 seconds then I simply threw an error. Lets look at the code for that. function waitForElement(selector, timeout = 10000) { return new Promise((resolve, reject) => { const interval = 100; const endTime = Date.now() + timeout; const check = () => { const element = document.querySelector(selector); if (element) { resolve(element); } else if (Date.now() < endTime) { setTimeout(check, interval); } else { reject( new Error( `Element with selector "${selector}" not found within ${timeout}ms` ) ); } }; check(); }); } Now instead of using document.querySelector() use await waitForElement(). function waitForElement(selector, timeout = 10000) { return new Promise((resolve, reject) => { const interval = 100; const endTime = Date.now() + timeout; const check = () => { const element = document.querySelector(selector); if (element) { resolve(element); } else if (Date.now() < endTime) { setTimeout(check, interval); } else { reject( new Error( `Element with selector "${selector}" not found within ${timeout}ms` ) ); } }; check(); }); } async function extractLinkedInData() { try { const nameElement = await waitForElement("h1"); const bioElement = await waitForElement(".text-body-medium"); const locElement = await waitForElement( ".text-body-small.inline.t-black--light.break-words" ); const followersElement = await waitForElement( '.artdeco-card .rQrgCqdAxxLhIAcLhxdsifdagjxISOpE span' ); const aboutElement = await waitForElement( ".JSzNEGEyfojpwDGomLCFeVXPtVfgJfKE span" ); const connectionElement = await waitForElement( ".OnsbwwsPVDGkAkHfUohWiCwsWEWrcqkY " ); const followerCount = parseInt( followersElement.innerText .replace(/,/g, "") .replace(" followers", "") ); const user = { name: nameElement.innerText, bio: bioElement.innerText, location: locElement.innerText, followerCount: followersElement.innerText .replace(/,/g, "") .replace(" followers", ""), about: aboutElement.innerText, url: document.URL, }; const connectionText = connectionElement.innerText.replace("\n", ""); if (/\d+([+])? connections/.test(connectionText)) { user.connectionCount = 501; } else if (/\d+([+])? connections/.test(connectionText)) { const connectionCount = parseInt( connectionText.replace(" connections", "") ); user.connectionCount = connectionCount; } console.log(user); chrome.runtime.sendMessage({ type: "user_data", data: user }); } catch (error) { console.error("Error fetching elements:", error); } } extractLinkedInData(); Now the only thing that is left to do is to perform a POST request. I tried to do that just after getting all the tags in the extractLinkedInData()just to realise that you cannot do things like perform API calls or HTTP requests from the client console on some pages. LinkedIn does not allow this, so we have to figure out another way to do this. We send a runtime message using Chrome API and listen for this in background.js and perform the POST request from there. // background.js chrome.runtime.onMessage.addListener(function (message, sender, sendResponse) { if (message.type === "user_data") { fetch("http://localhost:5555/getinfo/", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(message.data), }) .then((response) => response.json()) .then((result) => { console.log("Success:", result); }) .catch((error) => { console.log("Error:", error); }); } }); And with that we're done! The last part where we performed the POST request from background.js rather than content.js was similar to the problems I faced in a pervious article on electron.js where certain scripts do not have access to node packages. That article covers how to read data from the serial port in a desktop application. If you know a better way to do something similar let me know in the comments👇🏻

Voxa - Our Journey from Idea to Implementation

Aryan — Fri, 17 May 2024 01:26:23 GMT

A while back, Prakriti and I participated in Women Techies, a hackathon at our university organized by the Google Developer Student Clubs at VIT Vellore. Our goal was simple: reach the final pitches and present on stage. Our journey began with a flurry of ideas as Prakriti and I brainstormed potential projects. We wanted to create something impactful, something that could stand out among the plethora of brilliant concepts around us. After several rounds of discussion and plenty of coffee, we finally landed on an idea that sparked our enthusiasm. We decided to make a command line tool for developers—a project that would give us a break from developing solutions for others and allow us to create something we would actually use. Developing a tool to solve a problem we personally faced made the process both easier and more enjoyable. Knowing the ins and outs of the issue meant we could design a solution that truly met our needs, making the project not just a task but a passion-driven venture. As usual, we started by laying out the basic blueprint of our project, refining the idea as we progressed. We carefully selected a few key features and decided on the tech stack we would use. While GoLang or Rust would have been ideal for building a CLI, we opted for JavaScript, a language we were both comfortable and proficient with. This choice allowed us to focus on delivering a robust and functional tool without the added hurdle of learning a new language under time constraints.The hackathon began, and the clock started ticking. We jumped right into programming and quickly completed two of the easier features while also setting up the GitHub repository. Neither Prakriti nor I are big fans of frontend development, so we were relieved that this project required no Figma designs or frontend coding. This allowed us to focus entirely on the backend and functionality, which played to our strengths and made the development process more enjoyable. As Review 1 approached, we decided to stop working on newer features and instead focused our efforts on polishing and refining the ones we had already completed. Our aim was to ensure that they were not only presentable but also free of bugs. One of my favorite practices during hackathons is to create a concise document that encapsulates the essence of our idea in a few bullet points, along with bullet points on what parts of the code we have completed and what we will be working on after the review. This document serves as a handy reference for ourselves and for any potential reviewers, providing a clear overview of our progress and direction. Since reviewers do not have a lot of time on their hands and teams spend a lot of time on presentations this gives them exactly what they want to know to look at while you present. After review 1 we hit a roadblock. We had spent too much time on trying to develop features and we were no close to completing them than we were a few hours ago. This is where most teams in hackathons fail. Failing to build something they planned on or taking too long and not being able to complete it in the give time frame is something which we generally struggle with. Sometimes it makes sense to think of something new instead of sticking to the same thing you've been trying to do considering the ideating phase in hackathons is short. We were tired, frustrated and this was when we felt like we would not achieve our goal of reaching the final pitches. We thought for some time and decided to scrap those features and started brainstorming features that we could replace them with. Now we had something to look forward to. We weren't stuck anymore! This was around the same time when the GDSC organisers had started jamming. We decided to add an agile tracker to our cli and a couple more features. We sat overnight and completed a major chunk of the newer features and took a small nap. After waking up with a fresh mind fueled with coffee we got back to programming. We finished the tasks with the highest priority first. Since review 2 was sneaking up on us and a requirement for it was to have a presentation we got to making that as well on the side. Our designing skills aren't really the best so we had to get some inspiration from here and there but 30 minutes later we had a decent presentation. Review 2 went well - all of our features worked well and there seemed to be no bugs. The only part that was left was a bit of documentation. Technically speaking the hacking time was over and now we just had to wait for the results to see if we made it to the final pitches. We were pretty tired so instead of finishing the documentation we took a pizza break instead. 2 hours later the results of the top 10 teams were up. There we were on screen! The debugging ducks made it to the final pitches! As the second-to-last team to pitch, we had the opportunity to witness the impressive projects developed by the other teams. Each presentation showcased a unique blend of creativity, technical prowess, and innovation. One project that particularly stood out was created by a team of freshmen who had developed an augmented reality app using Unity—a remarkable achievement considering the short 36-hour time frame of the hackathon. Soon after that results were announced. We learnt a lot from this experience. The biggest learning being that hackathons are not just about having the best programming team out there. A successful hackathon team includes skill, presence of mind, hardwork, and a good idea to begin with. Programming is just a way for you to bring your ideas to reality at a hackathon it doesn't matter if you have the best backend if your idea is generic. An unstable/hacky solution to a real problem has a better chance at winning rather than a really well programmed generic solution. Don't be afraid to change things mid way and try things out, that could be the difference between making it to the final pitches and getting eliminated in the final review. In a hackathon not too long ago a team changed their entire idea mid way and still managed to win in their track. Keep Hacking!

Object detection using OpenCV

K Akhilesh Saiju — Sun, 03 Mar 2024 17:21:11 GMT

In one of my previous articles, I elaborated on using OpenCV with python to switch on a user’s webcam, and to display the live feed. As a fun-ish Python project, I also created a GUI to pick out a spot in the feed and display its colour scheme, in RGB values. Scurrying around in the OpenCV documentation, I realized how little I knew, and how deep the proverbial rabbit hole goes, regarding this library’s scope of implementation. One of the topics that immediately piqued my curiosity was live object detection. Haar Cascades, an Introduction The one method that I kept seeing again and again, which is used hand-in-glove with OpenCV for object detection, is the use of Haar Cascades. A bit more primitive than more modern techniques, using Haar Cascades provides nigh-unparalleled speed, although with the caveat of frequent false-positives. Even when creating this project, I noticed that the algorithm had a strange obsession of claiming my neck as a second face. Despite its shortcomings, it’s still widely used, even while creating AI models. The method consists of an algorithm which can detect a particular type of object in real-time, based on pre-trained parameters. A Haar Classifier is used, which can detect features like edges and lines, and determines whether there is a particular object in the video, and can demarcate in what region it is. For example, in terms of face detection, the HaarClassifier will be able to differentiate eye and nose regions by comparing the brighter and darker pixels. The Classifier works by using similar patterns and comparisons. Creating a Face Detection Project in Python Well, there’s no better way to learn than diving off of the deep end. So, I decided to borrow what I learnt from my last project and add a face detection functionality to the GUI. First off, installing all the necessary packages, via the terminal: pip install opencv-pythonpip install Pillowpip install tk And subsequently, importing all the aforementioned packages: import cv2 from tkinter import * from PIL import Image, ImageTk Using OpenCV’s method to access the devices webcam: vc = cv2.VideoCapture(0) A fact that I failed to mention in my previous article, is that the parameter “0” is passed into the method to access the webcam. The address for any required video, from the file directory can also be passed as a parameter, to load the video vc = cv2.VideoCapture("Jack Ryan.mp4") Loading the Haar Cascade Classifier: face_classifier = cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_frontalface_default.xml") In this step we are using an XML file, which contains the instructions for the Haar Classifier. Here’s a snippet of the file: From within the file we can modify the parameters as per our requirements. This particular Classifier facilitates detection of front-facing portraits. Now, I have to make it absolutely clear, that there is a method built into OpenCV which displays the video feed, which runs indefinitely, until the user presses a specified key. while True: success, vf = vc.read() if (success == False): break gray = cv2.cvtColor(vf, cv2.COLOR_BGR2GRAY) fs = face_classifier.detectMultiScale(gray, 1.1, 5, minSize=(40, 40)) for (x, y, w, h) in fs: cv2.rectangle(vid, (x, y), (x + w, y + h), (0, 255, 0), 4) cv2.imshow("Face Detection", vf) if cv2.waitKey(1) & 0xFF == ord("q"): break vc.release() cv2.destroyAllWindows() It’s a neat method. However, sometimes, a person just has to flex their Tkinter GUI skills. Using the code from my last python project to integrate the feed into a GUI; window = Tk() window.title("Video feed") window.geometry("1200x700") button = Button(window,text="Click to detect face") button.grid(row=0, column=0) label = Label(window, height=720, width=1280, text="Video feed here") label.grid(row=1, column=1, columnspan=1) def show_frames(): success, vf = vc.read() if (success==False): return gray = cv2.cvtColor(vf, cv2.COLOR_BGR2GRAY) fs = face_classifier.detectMultiScale(gray, 1.1, 5, minSize=(40, 40)) for (x, y, w, h) in fs: cv2.rectangle(vf, (x, y), (x + w, y + h), (0, 255, 0), 4) frame = cv2.cvtColor(vf, cv2.COLOR_BGR2RGB) img = Image.fromarray(frame) imgtk = ImageTk.PhotoImage(image = img) label.imgtk = imgtk label.configure(image = imgtk) label.after(20, show_frames) button.config(command=show_frames) window.mainloop() In both methods, the frames from the feed are being converted to grayscale, and passed into the classifier. gray = cv2.cvtColor(vf, cv2.COLOR_BGR2GRAY) fs = face_classifier.detectMultiScale(gray, 1.1, 5, minSize=(40, 40)) Using another OpenCV method, a rectangle is mapped onto the frame, where the classifier has detected the face. for (x, y, w, h) in fs: cv2.rectangle(vf, (x, y), (x + w, y + h), (0, 255, 0), 4) Using the program to detect a face I tried loading a clip from Patriot Games (fantastic film, by the way) into the face detection model. The model clearly has no problem detecting dear old Dr. Jack Ryan. The model can also demarcate multiple faces, both in a video clip, and live webcam feed.

Kubernetes: The only Intro you will need

Devansh Agarwal — Fri, 09 Feb 2024 13:16:21 GMT

What is Kubernetes? Kubernetes also abbreviated as k8s is an orchestration tool allowing you to run and manage your container-based workloads. If you do not know what are containers, please check out this article first before carrying on with this article. What I mean by orchestration is the automated management, coordination and deployment of various containers, microservices and other resources. This ensures scalability, agility and flexibility of the application. Kuberenetes is mainly used for deploying cloud native applications and microservices. How does Kubernetes achieve this? Kubernetes can divide its architecture into two parts, cloud side and customer side. The cloud side of kubernetes consists of the Kubernetes API and the master node which communicates and keeps in sync with the customer side. It also allows us to define how to run our workloads. The customer side is the part users interact with if they decide to include kubernetes in their project. It consists of worker nodes and a very important part, kubelet which is reponsible for scheduling and making sure that the apps stay running healthily within the worker nodes. Each worker node has one kubelet service. The basic structure of a kubernetes cluster is a bunch of nodes known as worker nodes on the customer side which run pods, a small logical unit which runs the containers for our applications on the worker nodes, and each pod can contain multiple containers depending on if the services are tightly coupled to be deployed on the same pod or loosely coupled to be deployed on different pods. Kubernetes Components - API server - It is a component of kubernetes control plane that exposes the Kubernetes API. Scheduler - It assigns newly created pods to nodes based on resource availability, constraints, and workload requirements. Controller Manager - It manages various control loops that regulate the state of the cluster. etcd - It is a key-value store which stores cluster's configuration data, state and metadata. Kubelet - They run on each node and ensure that node's containers run healthily based on the pod's specifications. Kube Proxy - It mantains the network rules and performs packet forwarding for pods on the node. Now that we know of the different components that make up kuberenetes architecture, we should get to know the way in which we can use these components to take advantage of the services provided by kubernetes. Kubernetes allows us to access these components through the use of YAML files. (Fun Fact: YAML stands for YAML ain't markup language, this is the example of a recursive acronym). Types of Resources - There are many kinds of resources defined by Yaml file - Pod Deployment Service ConfigMap Secret Pod Resource - Below is an example of a Pod resource file - # A sample yaml file of kind pod apiVersion: v1 kind: Pod metadata: name: my-pod labels: app: my-app spec: containers: - name: my-container image: nginx:latest ports: - containerPort: 80 As you can see in this yaml file, we have specified the kind as pod meaning this yaml file is for configuring a single pod. Let's break down the fields in this file - apiVersion: Specifies the version of the Kubernetes API being used. In this case, it's v1. kind: Specifies the Kubernetes resource type. Here, it's a Pod. metadata: Contains metadata about the Pod, such as its name and labels. name: Specifies the name of the Pod. labels: Specifies labels for the Pod, which can be used for grouping and selecting Pods. spec: Specifies the specification for the Pod, including its containers. containers: An array of containers to be run in the Pod. name: Specifies the name of the container. image: Specifies the Docker image to use for the container. ports: Specifies the ports to expose on the container. containerPort: Specifies the port number on which the container listens. This is a basic example of how a yaml file would look like in a project. Deployment Resource - Now that you have seen this pod resource type, do you think it addresss all the issues an application may face like deployment, scalability and flexibility? The answer to this question is NO. This resource only addresses the deployment issue but we still need to make the application scalable and flexible. For this, kubernetes defines another resource type known as Deployment. This is an abstract resource which allows us to achieve the desired state for our application. Through this we can specify the number of pods we need replicate and maintain at all times. This means, if a certain pod dies due to a reason, it will automatically create a new pod in order to replace this dead pod. Through deployment resource, we can also specify the strategy for rolling out our applications thus making our application scalable and flexible. Here is a basic Deployment resource file, - # A sample yaml file of kind deployment apiVersion: apps/v1 kind: Deployment metadata: name: my-deployment spec: replicas: 3 selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: my-container image: nginx:latest ports: - containerPort: 80 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 In this, we can see the kind as Deployment, therefore this is a Deployment resource type. Now I will define all the new fields which can be seen in our Deployment file - apiVersion: Specifies the version of the Kubernetes API being used (apps/v1 for Deployments). kind: Specifies the Kubernetes resource type (Deployment). metadata: Contains metadata about the Deployment, such as its name. spec: Specifies the specification for the Deployment. replicas: Specifies the desired number of replicas (instances) of the application to run (in this case, 3 replicas). selector: Specifies the labels used to select the Pods to manage. matchLabels: Specifies the labels that Pods must have to be managed by this Deployment. template: Specifies the template for creating Pods managed by this Deployment. metadata: Contains metadata for the Pods. labels: Specifies the labels to apply to Pods created from this template. spec: Specifies the specification for the Pods. containers: Specifies the containers to run in the Pods. name: Specifies the name of the container. image: Specifies the Docker image to use for the container. ports: Specifies the ports to expose on the container. containerPort: Specifies the port number on which the container listens. strategy: Specifies the deployment strategy. type: Specifies the type of deployment strategy (RollingUpdate). rollingUpdate: Specifies parameters for the rolling update strategy. maxSurge: Specifies the maximum number of additional Pods that can be created during the rolling update. Service Resource - There is one problem with the setup till now, that is the ability to access these deployed pods. After deployment, each of these pods arte assigned an internal ip address which needs to be hit in order for us to access these pods. But the problem is, these ips are not static and can change with subject to availability. Here kubernetes provides us with a solution using a resource known as Service which can group a set of pods or deployments together and assign it a static ip allowing easier access. Services are of many types - ClusterIP - Default type. It provides the Service an internal ip through it can be accessed. NodePort - Exposes the service on each node's ip at a static port. LoadBalancer - Exposes the service externally using a cloud provider's load balancer Below we observe an example of NodePort service type - # A sample yaml file of kind service apiVersion: v1 kind: Service metadata: name: my-service spec: selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 8080 type: NodePort You can observe that the kind of yaml file is Service and the type specified in the spec is NodePort. apiVersion: Specifies the Kubernetes API version being used. kind: Specifies the type of Kubernetes resource, which in this case is a Service. metadata: Contains information about the Service, such as its name. spec: Defines the specification for the Service. selector: Specifies which Pods the Service should target. In this case, it targets Pods labeled with app: my-app. ports: Specifies the ports that the Service will listen on and how it will route traffic to the target Pods. protocol: Specifies the protocol used for the port (e.g., TCP, UDP). port: Specifies the port on which the Service will listen. targetPort: Specifies the port on the Pods to which the Service will forward traffic. In this example, since the selector selects pods with the label app: my-app, the pods we created in the deployment files will be selected and grouped together under this service my-service. Now we can take advantage of another in-built service of kubernetes, kubeDNS to directly access the pods using the service name, since the DNS service resolves it to the pods ip address. This is very useful when buiding microservices as you would often find the need to call other services present in the cluster. Ingress Resource - Ingress is an API object used to manage external access to the services within the cluster. It allows you to define the routing rules for the traffic to different services based on the endpoints. It supports HTTP and HTTPS. An example of Ingress can be seen below - # This is a sample yaml file of kind Ingress apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-ingress spec: rules: - host: example.com http: paths: - path: /app pathType: Prefix backend: service: name: my-service port: number: 80 apiVersion: Specifies the Kubernetes API version being used, which in this case is networking.k8s.io/v1. kind: Specifies the type of Kubernetes resource, which is Ingress. metadata: Contains information about the Ingress resource, such as its name. spec: Defines the specification for the Ingress. rules: Specifies a list of host rules and their associated paths. host: Specifies the hostname for which the Ingress rules apply. In this example, it's set to example.com. http: Specifies that this rule is for HTTP traffic. paths: Specifies a list of paths and how to route traffic for each path. path: Specifies the URL path to match. In this case, it's /app. pathType: Specifies the type of matching for the path, which can be Prefix, Exact, or ImplementationSpecific. Here, it's set to Prefix, meaning that any path that starts with /app will match. backend: Specifies the backend service to forward matching requests to. service: Specifies the name of the Kubernetes Service to forward traffic to, which is my-service in this example. port: Specifies the port number of the backend service to forward traffic to. Here, it's set to 80. An ingress does not actually perform any routing itself, you need an Ingress Controller to interpret the ingress object and route the traffic. Popular controllers include - NGINX Ingress Controller, Traefik, HAProxy Ingress. ConfigMap - Sometimes when you are developing an application, there are certain values that you do not know during development and can only be defined during runtime. These variables are environment variables which contain important configuration data for the application api key, datable url etc. To manage these variables efficiently, kubernetes provides a way in a resource known as ConfigMap which stores these environment variables. Example of ConfigMap - # This is a sample yaml file of kind ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: my-configmap data: DATABASE_URL: "mysql://username:password@hostname:3306/mydatabase" API_KEY: "abcdef123456" LOG_LEVEL: "DEBUG" apiVersion: Specifies the Kubernetes API version being used, which is v1 for ConfigMaps. kind: Specifies the type of Kubernetes resource, which is ConfigMap. metadata: Contains information about the ConfigMap, such as its name. data: Contains key-value pairs representing the configuration data. Each key-value pair represents a configuration entry. The keys represent the names of configuration parameters. The values represent the corresponding configuration values. As you can see, the environment variables are stored in key value pairs making it easy to understand their values. In this way, we can easily change the values of environment variables whenever we want. Secret Resource - There are certain variables which contain sensitive data like admin username and password which cannot be leaked to anyone. These cannot be stored in a config map, but a different resource known as Secret. This ensures the protection of this sensitive information. Below is the a Secret Resource File - # This is a sample yaml file of kind Secret apiVersion: v1 kind: Secret metadata: name: my-secret type: Opaque data: username: YWRtaW4= password: MWYyZDFlMmU2N2Rm apiVersion: Specifies the Kubernetes API version being used, which is v1 for Secrets. kind: Specifies the type of Kubernetes resource, which is Secret. metadata: Contains information about the Secret, such as its name. type: Specifies the type of data stored in the Secret. In this case, it's Opaque, which means arbitrary data. data: Contains the actual secret data, encoded as base64. Each key-value pair represents a piece of sensitive information. The values are base64-encoded representations of the actual secret data. These values are base-64 encoded in kubernetes storage which allows for secrecy. Now we have learned about a lot of resources, it is time to learn about a tool in kubernetes which can use these yaml files to spin up pods in the cluster. Kubectl: A command-line tool kubectl is a CLI tool provided by kubernetes in order to interact with the cluster and make changes in it. It allows users to perform various operations on Kubernetes resources, such as creating, updating, deleting, and managing pods, deployments, services, and other Kubernetes objects. When you make a project and want to deploy it on a kubernetes cluster using the yaml files decribed in the previous section, by convention you make a manifestsdirectory in the root path of your project and write all the yaml files in there. After which, using a command line, you enter the manifests folder and write the command - kubectl apply -f ./kubectl apply as decribed above applys the resources according to the configuration yaml files. The -f flag allows you to specify the directory where you want the kubectl apply to pick configuration files from and ./ means the present directory which is the manifests directory. You can also write something like this - kubectl apply -f file1.yaml -f file2.yaml which applies configurations from both files. The kubectl get command is very useful if you want to get some information on a particular resource, for example node or deployment or pod. So command would look like this - kubectl get resource-name -o wide This gets all the resources of this type in your cluser. The -o wide flag tells kubectl to output additional information about the resource in a wide format. If you want to delete the resources you created with the configuration files, you can type - kubectl delete resource-name to delete the resource completely. kubectl scale command allows you to scale a deployment like - kubectl scale --replicas=3 deployment/my-deployment This makes 3 replicas of a deployment knows as my-deployment. Another very used command is kubectl exec which essentially just executes any command you want inside a container in the cluster. For example - kubectl exec -it my-pod -- /bin/bash executes a command /bin/bash inside a pod, my-pod in interactive mode specified by the -it flag. With this, we end the article on Kubernetes, hope you learnt a lot and make sure to apply it in your future projects.

Communicating via the SerialPort using Electron.js

Aryan — Mon, 05 Feb 2024 10:03:59 GMT

The serial port is like a digital messenger that electronic devices use to share information. If you have ever workeded with an Arduino or Raspberry Pi, you may know that we create a serial connection, and begin sending information over a specific baudRate. The outputs are generally viewed on a serial monitor. Now we being computer science students must know how to integrate this hardware with software as well. How can we use the data being sent by the Arduino or Raspberry Pi inside a script, webb application, mobile application or a desktop application. For this multiple languages and frameworks support communication via the SerialPort, however using these is troublesome for beginners at first as we will see in the article. Programming the microcontroller Let us begin by first writing a simple script which sends data over the serial port form the microcontroller. For this you will need a microcontroller, in my case I am using an Arduino UNO, the cable that connects the microcontroller to the laptop and the Arduino IDE, you can also use the Arduino Web Editor if you do not wish to install the IDE. We will be writing a simple code which does the following: Creates a serial connection Continuously sends data over the connection // main.ino void setup() { Serial.begin(115200); // baudRate } void loop() { int randomNumber = random(100); // generates random number Serial.println(randomNumber); // prints to serial monitor delay(100); // pause for 100 miliseconds } Now upload it to the microcontroller using the upload button on the top left corner and see the results on the serial monitor. If the serial monitor is not visible then click the button on the top right corner which looks like a magnifying glass. Okay so now that we have setup the script for the microcontroller lets try to read it. Using javascript to read from serialport So if you try to find the code on ChatGPT you find the following code, where you will have to just replace the portName and baudRate and it should work just fine. // main.js const SerialPort = require('serialport'); const Readline = require('@serialport/parser-readline'); const port = new SerialPort('COM8', { baudRate: 115200 }); const parser = port.pipe(new Readline({ delimiter: '\n' })); parser.on('data', (data) => { console.log('Data from serial port:', data); }); port.on('error', (err) => { console.error('Error:', err.message); }); The first thing that you realise after running this is that something is wrong because this isn't working?! Note you will need npm and install the serialport module using the command npm install serialport. To run the javascript file simply use node /path/to/js-file in the terminal. In my case it was node main.js. What do you even mean by SerialPort is not a constructor?? These kind of error arise due to the change in the way the constructor is called which was probably brought in, in a later version of the package. So lets see the changes you'd have to do to this code to get it to work. // main.js const { SerialPort } = require('serialport'); // instead of const SerialPort = require('serialport'); const port = new SerialPort({ path: 'COM8', baudRate: 115200 }); // note that all params are passed // as an object to the SerialPort // constructor So it should work now right? No its not going to work that easily 😂😭 The next error you're going to get it with the ReadLine constructor. It would look somewhat like this. So again this is an error with the declaration of the ReadLine module and the way the constructor is called. So looking at the docs now the correct syntax is as follows. // main.js const { SerialPort } = require('serialport'); const { ReadlineParser } = require('@serialport/parser-readline') const port = new SerialPort({ path: 'COM8', baudRate: 115200 }); const parser = port.pipe(new ReadlineParser({ delimiter: '\r\n' })) parser.on('data', console.log) So now it should surely work right? I mean its just a 5 line code, what could go wrong? So lets run it. Okay so we have not encountered that type of error before. This error has arisen because the access to the serialport was denied to our code. This is a very beginner mistake that almost everyone makes and it is a very simple error to solve. When we ran the code in the Arduino IDE we used the serial-monitor to see the output. Only one process can have access over the serialport at a time, and since the access was with the serial-monitor we got this error. So now if we go back and close the serial-monitor and then run our code using node main.js we shoult be able to see the desired outputs. But why stop here. Lets try to use this 'working' code in a desktop application. Electron Electron.js is a javascript framework designed to create cross platform desktop applications using web technologies like HTML/CSS/JS that are rendered using a version of the chromium browser engine and a backend using the node.js run-time environment. Let us try to build a basic desktop application which will listen over a serialport at a specific baudrate and display the data in our application. Setup The setup of the application is pretty simple. Run the following commands. npm init -y npm install electron serialport In package.json add "start": "electron .", to scripts Replace index.js with main.js in the main section in package.json touch main.js renderer.js index.html style.css Creating main window Paste the following code inside the main.js file. This simple creates a window of size 500x600 and it's a standard boilerplate template. // main.js const { app, BrowserWindow } = require("electron"); const path = require("path"); const isMac = process.platform === "darwin"; function createMainWindow() { const mainWindow = new BrowserWindow({ title: "Propulsion Testing Software", width: 500, height: 600, }); mainWindow.loadFile(path.join(__dirname, "index.html")); mainWindow.maximize(); //mainWindow.setFullScreen(true); } app.whenReady().then(() => { createMainWindow(); app.on("activate", () => { if (BrowserWindow.getAllWindows().length === 0) { createMainWindow(); } }); }); app.on("window-all-closed", () => { if (!isMac) { app.quit(); } }); Now let us write the HTML and CSS code to structure and style the page. Serial Reader

Serial Reader

This data is being streamed from the serial port:

/* style.css */ body{ background-color: rgb(62, 62, 62); display: flex; flex-direction: column; align-items: center; justify-content: center; } .content{ background-color: black; margin-top: 20%; height: 50vh; width: 70%; border-radius: 10px; color: white; padding: 20px; } .serial-content{ background-color: rgb(95, 95, 95); width: 40%; height: 40px; border-radius: 10px; color: black; } #serial-content{ padding-left: 20px; padding-top: 10px; font-size: 20px; font-weight: 600; color: white; } With this we have completed the basic setup of a basic electron.js application. Renderer.js Now we should be able to simply copy paste the code written here inside the renderer.js file and ideally it should work. Let us try that. Paste the code below inside renderer.js // renderer.js const { SerialPort } = require("serialport"); const port = new SerialPort({ path: "COM8", baudRate: 115200 }); const serialDisplay = document.getElementById("serial-content"); window.addEventListener("DOMContentLoaded", function () { port.on("readable", function () { const data = port.read().toString(); console.log("Data:", data); serialDisplay.textContent = data; }); }); Now let us run the application using npm start. What do we observe? There are no errors visible in the terminal however the code isnt working. If we open the dev-tools using Ctrl + Shift + Iwe can see that we have the following error. Okay so let us try to understand why we might've gotten this error. In electron.js there are 2 types of processes. The main process and a renderer process. The main process is the one that creates windows, under which multiple renderer processes can run. By default these renderer processes do not have access to using node modules out of the box. For this you will have to enable nodeIntegration in the main.js file like so. And this is what even ChatGPT would tell you to do. // main.js // ...rest of the code const mainWindow = new BrowserWindow({ title: "Propulsion Testing Software", width: 500, height: 600, webPreferences: { nodeIntegration: true, } }); // ...rest of the code Let us see if this works or not. Run the application again using npm start. It doesn't work?! If we enabled nodeIntegration we should be able to require('serialport') in renderer.js but that is not the case. Now ChatGPT would tell you to use something called the ipcRenderer and the ipcMainto solve this issue. IPC stands for inter process communication. Since we are unable to use node modules inside the renderer process it will suggest you to either read the data in your main process or setup a preload script for your renderer process. The main idea behind this approach being to read the data in one place and send the data over ipc to the desired file. You can try this approach however even this won't work immediately. Also at times you would not want to use ipc as that would involve streaming the data twice! As mentioned here we must set enableRemoteModule to true and contextIsolation to false. ContextIsolation allows us to build a contextBridge to enable certain functions and services to the renderer process from the preload script which it would not have access to by default. Since we won't be using this we will set it to false. // main.js // ...rest of the code webPreferences: { nodeIntegration: true, contextIsolation: false, enableRemoteModule: true, } // ...rest of the code Now when we run the application it should work perfectly!👌 Embedded media If you face any other errors apart from the above mentioned, do let me know in the comments below👇

What is Bootstrap and how to implement its components.

Atulya — Sat, 27 Jan 2024 16:27:16 GMT

What is Bootstrap? All of us in the CS and IT domain are very much familiar with the term "Web development". It basically means to develop a webpage or a website. Web development on a bigger scale contains two main parts i.e. Front End and Back End. Bootstrap is a front end framework that is created by Twitter. It helps in the creation of mobile first and responsive websites. It contains a collection of HTML, CSS, and JavaScript elements, including buttons, forms, navigation bars, and more, to make the process of designing a unified and eye-catching user interface easier and faster. STARTING TEMPLATE OF BOOTSTRAP: Bootstrap as mentioned above is a front end framework, so in order to get started with it we need to install or use any online editor that supports HTML, CSS and JavaScript. I have personally used Visual Studio Code for understanding and implementing this framework. The start template of the code is very simple where we will learn how to print "Hello World" on a web page using HTML. Getting Started with BootStrap

Club TechKnowHow!

Try using the above snippet on your desired editor. The above code is a just a simple HTML code through which you can easily start Bootstrap. Few components of Bootstrap: 1.NAV BAR Nav bar is one of the basic and most important components of Bootstrap without which you cannot make any desired webpage. It is one of the block units of a webpage that makes it look complete. It contains the functionality of toggling, collapsing and drop divider ,etc. We can implement the Nav Bar component in your webpage by simply using the nav class tag. Here is an example of all the sub-components included in a responsive light-themed navbar that automatically collapses at the lg (large) breakpoint. By using the above given code you can add navigation point and bars to your webpage and can make it look proper and complete. This example uses background (bg-body-tertiary) and spacing (me-auto, mb-2, mb-lg-0, me-2) utility classes. When you implement the above code you would get a document which would look something like this. 2.CAROUSEL The carousel is a slideshow that uses a little bit of JavaScript and CSS 3D transforms to cycle between various pieces of material. It can operate with a collection of pictures, text, or unique markup. Support for previous and next buttons and indicators is also included. We can implement the Carousel component in any webpage which will feature a slideshow of all the desired images which is Autoplayed on the web page. The snippet of Carousel component along with Nav bar is given below:

Welcome to Coder.com

Technology,development and skills

The most trusted coding blog

Technology,development and skills

Award winning coding blog

Technology,development and skills

This Carousel component basically adds features to your webpage which makes it look attractive. It creates a skeleton framework of the website. By using this code you will get the following results in addition to the earlier component on your webpage. 3.BUTTON FUNCTION We can use Bootstrap’s custom button styles for actions in forms, dialogs, and more with support for multiple sizes, states, and more. In addition to this, Bootstrap has a base .btn class that sets up basic styles such as padding and content alignment. By default, .btn controls have a transparent border and background color, and lack any explicit focus and hover styles. We have implemented the Buttons component in the given snippet: We can also change the color of the buttons in the same way as we have done above, by using the keywords danger(for red), success(for green) and primary(for blue). The above code will give the following results on the webpage. 4.CARDS COMPONENT The cards feature of Bootstrap offers a versatile and expandable content container with several variations and configurations. Cards component can easily be implemented and gives a subtle look to a blogging website. We have implemented cards in two places in our code. 1.World and Design feature

World

Featured post

Nov 12

This is a wider card with supporting text below as a natural lead-in to additional content.

Design

Post title

Nov 11

This is a wider card with supporting text below as a natural lead-in to additional content.

The above code will give the following output: The Card component can be used to indicate something on the website. In the above syntax we have used two cards names WORLD and DESIGN which contains some information and images which were used in the program by using the basic img class. 2.Featured Card Component The featured card was used in the webpage by using the given syntax:

Featured

Special title treatment

With supporting text below as a natural lead-in to additional content.

Go somewhere

2 days ago

The above code will give the following output: This adds a featured card component at the end of the web page which completes the overall look of the webpage. 5.MODAL We can use Bootstrap’s JavaScript modal plugin to add dialogs to your site for lightboxes, user notifications, or completely custom content. It adds a dialogue box to the nav bar buttons which can be used to show a particular prompt.NOTE: Bootstrap only supports one modal window at a time. Nested modals aren’t supported as we believe them to be poor user experiences. 6.FOOTERA footer is an additional navigation element that may contain a variety of content, such as links, buttons, company information, forms, and more. For instance, a basic footer might include text, links, and a copyright section. To customize the appearance of the footer, you can adjust the background color using a CSS class like .bg-body-tertiary. The syntax for adding a footer is given below: The result of the above code will be: The final webpage after using all the components would look something like this: Hence, by using the above all components of Bootstrap you can make a very simple and usable website for any school or university related work. These components are the most basic and important components for Frontend, without which you would not be able to make a proper website. This article would provide a great start to your Frontend journey in web development by providing you all the components and functions of Bootstrap.

Git and GitHub - How to Setup and Get Started

Vaidehi — Fri, 19 Jan 2024 11:42:04 GMT

Topics discussed What is Git? Git terminology Git Installation and setup What is GitHub? Basic functionalities of GitHub using Git What is Git? Git can be described as a distributed version control system that tracks changes in any set of computer files, used mainly for coordinating work among programmers who are collaboratively developing source code during software development. It aims to support data integrity and non linear workflows. Another way of understanding it is that Git views data as a series of snapshots of a mini file system. With every commit (discussed in the next segment) of the state of your project files Git takes a snapshot of the current status of your files and stores a reference to that snapshot. Git Terminology Commit: This command captures a snapshot of the project's currently staged changes. Committed snapshots can be considered as “safe” versions of a project, they will not be changed unless explicitly stated. Repository: It is like a data structure used by VCS (version control system) to store metadata for a set of files and directories. It contains the collection of files as well as the history of changes made to those files. Repositories in Git can be considered to be a project folder. Branch: A branch is a version of the repository that diverges from the main working project. It is an essential feature available in most modern version control systems. A Git project can have more than one branch. Remote: The term remote is used with respect to the remote repository. It is a shared repository that all team members use to exchange their changes. A remote repository is stored on a code hosting service like an internal server, GitHub, Subversion and more. Origin: It is a reference to the remote repository from a project that was initially cloned. It is used instead of that original repository URL to make referencing much easier. Pull Request: The term Pull is used to receive data from GitHub. It fetches and merges changes on the remote server to your working directory. Pull requests are used by developers to notify team members that a feature has been completed and the team members need to review the code and merge it with the master branch. Push Request: Uploading local repository content to a remote repository. Pushing is an act of transferring commits from your local repository to a remote repository. Git Installation and Setup Step 1: Google git and click on the first search result that pops up which should be the git official website – Git SCM (https://git-scm.com/) and scroll till you see the download for windows button and click on it. Click on the ‘click here to download’ button to download it. Step 2: After downloading this executable file click on it to start the setup. The first window is about the public licence. Click on next to continue. Step 3: Next window is about the location where Git will be installed. Click on next if there is no good reason to change the location. Step 4: Next window is for selecting components. Here you can choose to add an additional icon by clicking the check box. Click on next to continue. Step 5: Here you can select the start menu folder. Leave it as it is and click on next. Here you can choose your default editor. I’ve chosen Visual Studio Code. Step 6: In the next window select the option to ‘override default branch name’ and give the name as ‘main’. This is because when you use Git with GitHub, the default branch name on Git is ‘main’. Step 7: Leave the settings in their default state for the next few windows till the window for ‘configuring experimental options’ is reached. Select both the available options and click on install. Step 8: After installation of all the files, check the box which says ‘Launch Git Bash’ and uncheck the other box and click on finish. This concludes the basic setup of Git. The next segments will elaborate on GitHub and how to use it using Git. What is GitHub? GitHub is an AI-powered developer platform that allows developers to create, store, and manage their code. It uses Git software, providing the distributed version control of Git plus access control, bug tracking, software feature requests, task management, continuous integration for every project. It is mainly used to hold open source software development and allows the tracking, storing and collaboration on software development projects. Basic functionalities of GitHub using Git Let's start from scratch here. If you've never used GitHub before, start by making a GitHub account by going to the GitHub official website and clicking the signup option and filling in the necessary details. Now that we have our account setup, let's go back to Git. Locate the Git folder on your PC. In it you will find some options to choose from like Git GUI, Git Bash etc. We will work with Git Bash. One of the first we must get into to get started is to commit a file of code. Follow the steps given below: Step 1: Locate whatever code you want to commit in your file explorer. Here I'm using an html file as an example. Right click on the file and click on ‘show more options’ and then click on Git Bash. This opens a terminal. Step 2: In the terminal the first command w indeed to give is the git init command which will initialise git in this folder. Step 3: Now if the ls command is given, a .git folder can be seen. Step 4: Now we can check the git status by giving the command git status. It will show some untracked files which are the files we are working with right now. Step 5: The next step is to add this file by giving command git add [file_name]. Now if git status is run, the files will be seen in green indicating that the files have been added to git but need to be committed. Step 6: Before committing our files we need to configure git. Do this by running two commands git config --global user.email “your email” and git config --global user.name “Your name”. Step 7: After configuring git and adding all the files, commit them using the command git commit -m “comment” [file_name] (you can give any comment). Now if we check git status it will show that there is nothing to commit. Step 8: Now going back to GitHub. Once on the website with your account, click on ‘new repository’ to make a new repository. Usually the name is the name of the main folder. Here it is ‘blog’. Add a description and you can also add the README file, licence etc. then click on create repository. Now we will create a GitHub token which will be required to push our code into the repository. Step 1: Click on your profile on the GitHub homepage and then go to settings. Step 2: Scroll to the bottom and find the ‘developer settings’ option. Step 3: Next click on personal access tokens and go to the ‘token (classic)’ option. Step 4: Next click on ‘generate new token (classic)’ , name your token and specify the expiration date. Step 5: Now under select ‘scope’ check all the check boxes and generate the token. Step 6: The token generated can only be seen once. Thus, it is a good idea to copy it and store it somewhere safe. Step 7: This token will now be used in the next command to be executed on Git Bash. Give the command git remote add origin https://[token]@github.com/[user_name]/[repo_name.git]. This command will add the remote origin of the current project. Step 8: Now give the push command git push -u origin main. This command will push your code into GitHub. Step 9: After this if you refresh your GitHub page you should be able to see your code files in your repository. This covers the basic commit and push operations on Git. Let us now see how we can make changes to the code file and update it in GitHub using Git. Step 1: Give the command ls to see all the files currently committed. Step 2: Next give the command code . which will open the code files in your chosen editor. (Here it is Visual Studio Code). Step 3: Make the required changes in the code and save. Step 4: Give git status command. It will show that there is some uncommitted code. Step 5: Commit this code by giving git commit -m “comment” [file_name] command. Step 6: Every time a code is committed, it needs to be pushed as well. So, push the code using git push -u origin main command and the changes made should be reflected in the repository on the GitHub website. With Git and GitHub in your arsenal, you're now equipped to track changes, collaborate effectively, and build a portfolio of your work. Don't hesitate to explore more advanced features, delve into online resources, and keep practising. The possibilities are endless, so keep learning, keep creating, and keep pushing your code.

Server-Side scripting with XAMPP

K Akhilesh Saiju — Tue, 16 Jan 2024 09:34:34 GMT

XAMPP is an open-source web server distribution which has become the de facto package used while handling PHP files. XAMPP stands for (X) Cross-platform Operating System, Apache, MySQL, PHP and Perl. Installing and setting up XAMPP The official XAMPP installer, provided by Apache can be found at https://www.apachefriends.org/download.html , for various operating systems. Installing and setting up the XAMPP control panel, the user can access it as such: The XAMPP Control Panel provides access the various components of the user’s server. Creating a Demo webpage To demonstrate the working of the server, we can create a simple webpage, using HTML and PHP, which can be accessed through the server. The “htdocs” folder can be found where XAMPP was installed. The files and content to be accessed in the web server need to be saved in this folder. Using HTML, we can create a webpage which utilizes a form to collect any arbitrary data from the user. In this case, a variety of buttons are created, each having its own value, as per the user’s preference. Image of an Egg

Welcome to Image of an Egg

Here's the aforementioned image

Get a personalized egg prepared for you

We can use PHP to create a webpage which accesses information provided by the user in the previous page and uses it to personalize the endpage. The PHP file accesses user information from the server-side, to create a better frontend experience. An anchor tag is also provided so that users can return to the original page. Document

Welcome,

Here is your egg.

I don't like it

Take me back to original egg

Starting the Web server and accessing created webpages The XAMPP Control Panel is used to start the Apache component. The XAMPP dashboard can accessed by typing http://localhost/dashboard in the address bar of any web browser. The created webpages can be accessed by typing the following format in the address bar: http://localhost/

Created websites, as accessed via the web server: Working in a MySQL database with XAMPP MySQL is one of the most popular database management systems, which can organize and store data of many types, such as text, numerical information, dates, and also JSON. A MySQL database can be easily built using the XAMPP Control Panel. The MySQL module has to be started, along with the Apache module. Use the Admin button in the MySQL to access the Admin page of the XAMPP server. Then navigate to the phpMyAdmin page. Create the MySQL database by choosing the Database option from the drop-down menu. In the Structure tab, the user can find the option to create a Table, with a specified name, number of columns and so forth. After creating the Table, the user can save the database, for future perusal and use. With a tool like XAMPP, the use of MySQL is further simplified, and accessibility is increased.

A Hands-on Approach to Docker

Devansh Agarwal — Sat, 13 Jan 2024 11:18:25 GMT

Table of Contents What is Docker? Installation and Setup Docker Containers and Images Dockerfile Caching Port Mapping in Docker Networking in Docker Volumes in Docker Environment Variables in Docker Docker Compose What is Docker? Docker is a platform designed to simplify the process of developing, shipping, and running applications. It leverages containerization technology, allowing developers to encapsulate an application and its dependencies into lightweight, portable containers. Installation and Setup Visit Docker Desktop site and choose the application based on your operating system. Download the installer and run it. Follow the installation instructions as given by the installer. After installation is complete, open your terminal and run docker version to verify. If the version of docker is displayed, it has been installed correctly. Image vs Container A docker image is a standalone, lightweight package that contains all your code required to run your service. It serves as a template for creating docker containers. Many images can be found pre-built on Docker Hub and can be directly downloaded onto your local device by running the following command - docker pull image-name To build your own docker image, you need a special file known as Dockerfile which contains all the information required by docker engine to build the image in a step-by-step manner. Once the Dockerfile has been written, image can be built by running the command - docker build your-dockerfile-directory To give the image a name, you can use the flag -t to tag it: docker build -t image-name . The dot specifies the directory your dockerfile is in. A docker container in a runnable instance of a docker image. It wraps the application in an isolated environment along with its dependencies and executes it. Containers are portable and can be moved between different environments. To start a container, you need to run the command - docker run --name container-name image-name If you do not include the --name flag then, docker assigns a name to the container itself. Flag -d can also be used to run a container in the background. So it will be like: docker run -d --name container-name image-name You can see the list of running containers using the command - docker ps in your terminal. How to write a Dockerfile? A dockerfile consists of a list of step-by-step instructions to build an image. Some of the keywords are- FROM - (used to pull a base image to build your own image on) WORKDIR - (used to specify the directory in which you want to store your files inside the container) RUN - (used to run a specific command while building the image) COPY - (used to copy files from the local device to the container) CMD - (used to execute the command given only once when a container has started) Below is a sample Dockerfile created for an express application - FROM node:20 WORKDIR /usr/src/app COPY package.json . RUN npm install COPY . . EXPOSE 4000 CMD ["npm", "run", "dev"] In this dockerfile - First, we get the base image of node version 20 from the docker hub. Then it that base image we set our working directory as /usr/src/app . Then we copy over the package.json app into our working directory. For those who do not have experience with javascript, the package.json contains all the information on our application and also have the list of dependencies. You will see that we run npm install after this, what it does is read the package.json file and install all the required dependencies for our application. Then we copy over the remaining files into working directory. Next step, we write EXPOSE 4000 meaning, we expose port 4000 on our docker container. This has to be done to tell docker to accept all requests on 4000(the port we are listening to in our application). After which CMD ["npm", "run", "dev"] is written which means our docker container will execute the command npm run dev after it starts. For those who want the view of package.json file - //package.json { "name": "docker-demo-app", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "dev": "nodemon index.js", "start": "node index.js", "test": "echo \"Error: no test specified\" && exit 1" }, "type": "module", "author": "", "license": "ISC", "dependencies": { "express": "^4.18.2", "mongoose": "^8.0.4", "nodemon": "^3.0.2" } } And this is index.js, the entrypoint of our app - //index.js import express from "express"; import mongoose from "mongoose"; import { createProduct, deleteProduct, getAllProducts } from "./controller.js"; mongoose.connect("mongodb://demo-mongodb-container:27017/Testing").then((data) => { console.log(`Mongodb connected with server: ${data.connection.host}`); }); const app = express(); app.use(express.json()); const router = express.Router(); router.route("/products").get(getAllProducts); router.route("/product/new").post(createProduct); router.route("/product/:id").delete(deleteProduct); app.use("/demo-api", router); app.listen(4000, () => { console.log("Server is running on Port 4000"); }); Here you can see that we listening on port 4000. In index.js, We create a products collection in mongodb which contains the field name, description and price. getAllProducts function fetches all the products from mongodb database. createProduct function create a new product. deleteProduct function deletes an existing product from mongodb database. Caching Notice how in the dockerfile we seperately copy the package.json file and rest of the files. This is to reduce the time taken to build our image again and again. In Docker, layer caching exists meaning each line in the Dockerfile is cached by Docker in layers one above the another and while rebuilding image, if a certain layer has not changed, the image layer is directly cached. This means that even if one layer is changed, all the layers stacked on top of it will also be removed meaning we would have to run them all again. That is why we have copied over the package.json seperately. Since during development, the dependencies will not change as often as the rest of the files which would mean the COPY package.json and RUN npm install layers would remain cached and only the commands below it would run if files are changed. This would reduce build times. Port Mapping in Docker Whenever we write a backend application application or use a local database, we connect them to a port on the localhost to listen to. If we containerize these applications, they would get seperated from the ports of localhost as containers are isolated environments. To prevent this, port mapping exists in docker which looks like -p port1:port2. This flag tells docker to forward all requests on port1 of the localhost of local device to port2 of the docker container. For example, in the above index.js, it listens to port 4000 therefore we can write -p 4000:4000 to tell docker that all requests on localhost:4000 are forwarded to container port 4000. We can include port mapping using the following commands - First we build our image using dockerfile just as given above. Then, we include the flag -p 4000:4000in our docker runcommand as following - docker run --name container-name -p 4000:4000 image-name Now, our containerized application can recieve requests from outside web on port 4000. Networking in Docker Networking in docker means the containers ability to connect and communicate with other docker and non-docker applications. There are primarily 6 types of existing network drivers - host, bridge, none, overlay, ipvlan and macvlan. Of these we will be discussing host, bridge and none networks. For the rest, you can refer to Docker Networking Docs. Bridge Network driver - This is the network used by containers by default unless otherwise specified. With these network settings, a bridge is formed between the container and the local device's network and requests on the local device can be forwarded to the container using port mapping as seen in the previous section. Host Network driver - In this type of network, the isolation between the docker container and the host is removed and all the ports of the host are accessible to the container. Meaning any request on the host's network can be listened to by the container. None Network driver - In this type of network, the container is completely isolated from the host and the other containers. No network exists for the container and it can only work internally inside the container. Now, in order to define your network using any of the pre-existing network - docker network create bridge demo-network This command defines a network named my-network with properties of a bridge network which can be used my different containers to communicate between each other. In the given index.js file above, on line 5, we can see the url of mongodb database as mongdb://demo-mongodb-container:27017. Here demo-mongodb-containeris the name of container having mongodb database This is possible if both containers have same network - docker run --name demo-app -p 4000:4000 --network demo-network demo-image This is the container for the js application with network demo-network, port mapping 4000:4000, name demo-app and image demo-image. For the database container - docker run --name demo-mongodb-container -p 27017:27017 --network demo-network mongo Name of this container is demo-mongodb-container, port mapping is 27017:27017 and network is demo-network. Since both containers are on the same network, we can reference them by their container name. Volumes in Docker Whenever a container is started and then stopped, all the content inside it is destroyed which is not good for a database since it is required to store data between restarts. Therefore to tackle this, docker has a feature known as volumes which can store data from the specified application and update in on restart. A volume can be setup using the command - docker volume create volume-name To use a volume along with the container, you can add the flag -v volume-name:directory-for-data, here the directory-for-data refers to the location where the data you need to store in the volume is. For example in a container of mongo image, the database data is stored in the directory /data/db. Therefore, to add volume to the above demo-mongodb-container, you can write the following commands - 1. docker volume create demo-mongodb-data 2. docker run --name demo-mongodb-container -p 27017:27017 --network demo-network -v demo-mongodb-data:/data/db mongo Here volume flag is demo-mongodb-data:/data/db where demo-mongodb-data is the volume previously created and /data/db is the directory in which the database data is stored. Now both the volume and the directory /data/db will be in sync, meaning if data is stored in /data/db it will also be stored in volume and on restart of container when /data/db is empty, it will recieve data from the volume hence mantaining peresistent data across restarts. The command to remove a volume is - docker volume rm volume-name. Environment Variables in Docker Often their are certain values which you cannot hardcode into the codebase because of sensitive information like database passwords, JWT secrets etc. For these environment variables can be used to enter these values into codebase only at runtime of docker container. For our example, we will use the index.js file. In this lets replace 4000 in the last three lines of index.js by process.env.PORT . This is syntax for environment variables by javascript. So it will look like this - app.listen(process.env.PORT, () => { console.log("Server is running on Port", process.env.PORT); }); Now, our app has port as an environment variable which can be given as argument in docker during runtime with the flag -e MY_VARIABLE=value. So the full command will be - docker run --name demo-app -p 4000:4000 -e PORT=4000 --network demo-network demo-app See here in the flag we have passed PORT=4000 meaning process.env.PORT equates to 4000 in our code. Docker Compose You may have noticed that to run an application with two or more services like our demo-app (consisting of js application and mongodb database) you need proportional amount of commands in terminal. Also the command gets more complicated as you add more attributes to the container. To tackle this problem, there is a feature known as docker compose where all the containers information can be written in one file and just one command is used to run all of them. All the information for docker compose is written in a file called as docker-compose.yaml For the above containers with all their attributes containing the js application and mongodb database, the docker-compose.yaml will be - #docker-compose.yaml version: "3" services: demo-app: build: ./ ports: - "4000:4000" environment: - PORT=4000 depends_on: - demo-mongodb-container demo-mongodb-container: image: mongo ports: - "27017:27017" volumes: - ./mongodb-data:/data/db In this file, we can see in services, the names of the two containers demo-app and demo-mongodb-container. The demo-app has keys build(to get path of Dockerfile), ports(for port mapping), volumes and environment. It also has additional attribute depends_on which tells docker to start this container only after the container demo-mongodb-container has started. Similarly, the mongodb container has keys image (to get the name of image), ports(for port mapping) and volumes (to store data). You might have noticed that their is no network attribute in the yaml file, that is because all the containers started from the same docker-compose file share the same network internally, therefore there is no need to mention the network name explicitly. So with this we end our article on Docker.

Creating a REST API using Node.js & Express.js

Aryan — Fri, 12 Jan 2024 18:40:11 GMT

Following up on the previous article where we discussed what RESTful APIs are and how to create one in python using Django, in this article we will go over how to create a connection with a MySQL database and then perform crud operations using the data streamed through the database using node.js and express.js. Let us go over how to perform this step by step by creating a REST API for course info. Creating a table in MySQL & populating it with data First create a schema and name it courses. Then create a table for all_courses and define the required fields. For this demonstration we will have fields for id, year, course_name, faculty and slot. To do so run the following script. -- Schema.sql USE courses; CREATE TABLE all_courses ( id integer PRIMARY KEy AUTO_INCREMENT, year VARCHAR(255) NOT NULL, course_name VARCHAR(255) NOT NULL, faculty VARCHAR(255) NOT NULL, slot TEXT NOT NULL ); Notice that the id field is a PRIMARY KEy and it auto-increments. We are not required to pass in the id parameter while pushing data into the table, the table with automatically assign it a value of the id of the previous entry plus one. The remaining fields must be passed when creating entries. Now let us populate this table with some dummy data. You can change the following data to create your custom entries. -- Schema.sql INSERT INTO all_courses (semester, course_name, faculty, slot) VALUES ('First','Biology101', 'Professor Smith', 'A1'), ('Second','Physics202', 'Dr. Johnson', 'B1'), ('First','Chemistry303', 'Professor Davis', 'C1'), ('Third','History404', 'Dr. Taylor', 'D1'), ('Fourth','Mathematics505', 'Professor Miller', 'E1'), ('First','English101', 'Dr. Brown', 'A1'), ('Second','ComputerScience202', 'Professor White', 'B1'), ('First','Psychology303', 'Dr. Martinez', 'C1'), ('Third','Economics404', 'Professor Turner', 'D1'), ('Second','Sociology505', 'Dr. Hill', 'E1'), ('Fourth','Philosophy101', 'Professor Carter', 'A1'), ('First','Music202', 'Dr. Reed', 'B1'), ('Second','PoliticalScience303', 'Professor Hall', 'C1'), ('First','Geography404', 'Dr. Allen', 'D1'), ('Second','EnvironmentalScience505', 'Professor Scott', 'E1'), ('Fourth','Art101', 'Dr. Garcia', 'A1'), ('First','Linguistics202', 'Professor Lee', 'B1'), ('First','Astronomy303', 'Dr. Adams', 'C1'), ('Third','Marketing404', 'Professor Wright', 'D1'), ('Third','Statistics505', 'Dr. Lopez', 'E1') Creating a connection with database Once we have created and populated the database with dummy data we can proceed to create a connection with our MySQL database. First create a new node project using the npm init command. This will initialize a new Node.js project and create a package.json file in the current directory. Now install the MySQL package for your node.js project using npm install mysql2. Now we can write a script to create a connection with our MySQL database. const mysql = require('mysql2'); const connection = mysql.createConnection({ host: 'localhost', user: 'root', password: 'password', database: 'database_name' }); connection.connect((error) => { if (error) { console.error('Connection Failed.', error); } else { console.log('Connected Successfully!'); } }); connection.end(); Now we could use this connection to query data from the database like so connection.query("SELECT * FROM all_courses") however we would be using a pool of connections instead of a single connection. const mysql = require('mysql2'); const connection = mysql.createPool({ host: 'localhost', user: 'root', password: 'password', database: 'database_name' }).promise() Notice the .promise(), it transforms the query method into a function that returns a Promise. This allows you to handle asynchronous operations more conveniently using promises or async/await syntax. createPool maintains a pool of connections that can be reused. This is beneficial in scenarios where there are frequent database operations, as creating and closing connections repeatedly can be resource-intensive. Writing a functions to perform crud operations To perform CRUD operations we will write these functions - getCourses(), addCourse(), updateCourse() and deleteCourse. async function getCourses() { const [rows] = await pool.query("SELECT * FROM all_courses"); return rows } async function addCourse(year, course_name, faculty, slot) { const [result] = await pool.query( `INSERT INTO all_courses (year, course_name, faculty, slot) VALUES (?, ?, ?, ?, ?)`, [year, course_name, faculty, slot] ); return result; } async function updateCourse(course_name, newFaculty, newSlot) { const [result] = await pool.query( `UPDATE all_courses SET faculty = ?, slot = ? WHERE course_name = ?`, [newFaculty, newSlot, course_name] ); return result; } async function deleteCourse(course_name, faculty, slot) { const [result] = await pool.query( `DELETE FROM all_courses WHERE course_name = ?, faculty = ?, slot = ?`, [course_name, faculty, slot] ); return result; } If you are confused why we have used ? to pass the arguments, instead of directly using the variables refer our article on SQL injections to see why we do this. Setting up the express application for get and post requests Install express using npm install express and import express, db.js the file where you wrote the functions to query data to and from the database and all the functions. const express = require('express') const db = require('./db.js'); const addCourse = db.addCourse; const getCourses = db.getCourses; const updateCourse = db.updateCourse; const deleteCourse = db.deleteCourse; app.use(express.json()) app.get('/allcourses', async (req, res) => { const courses = await getCourses() res.send(courses) }) app.post('/add', async(req, res) => { const { year, course_name, faculty, slot } = req.body const added_courses = await addCourse(year, course_name, faculty, slot) res.status(201).send(added_courses[0]) }) app.put('/update/:course_name', async (req, res) => { const course_name = req.params.course_name; const { faculty, slot } = req.body; try { const updatedCourse = await updateCourse(course_name, faculty, slot); res.send(updatedCourse[0]); } catch (error) { console.error(error); res.status(500).send('Internal Server Error'); } }) app.delete('/delete', async (req, res) => { const { course_name, faculty, slot } = req.body try { const result = await deleteCourse(course_name, faculty, slot); if (result.affectedRows > 0) { res.status(204); } else { res.status(404).send('Course not found'); } } catch (error) { console.error(error); res.status(500).send('Internal Server Error'); } }); Our REST API is ready and now you can integrate this in any javascript project. Note of caution When pushing code to production of a shared version control system make sure that your connection details to the database like the table, username, password are not accessible to the viewer. This can be done using dotenv. Simple install it using npm install dotenv. Import it into db.js like so const dotenv = require('dotenv') and configure it using dotenv.config(). Now you can create a .env file in the same directory and replace the host, user, password and database like so. const pool = mysql.createPool({ host: process.env.MYSQL_HOST, user: process.env.MYSQL_USER, password: process.env.MYSQL_PASSWORD, database: process.env.MYSQL_DATABASE }).promise() Now your database details won't be visible to people viewing your code.

Dynamic Connectivity Problem

Akshit — Wed, 10 Jan 2024 18:31:07 GMT

What is Dynamic Connectivity? In computing and graph, a dynamic connectivity structure is a data structure that dynamically maintains information about the connected components of graph. Dynamic Connectivity Problem Given a set of N objects. Union command: connect two objects. Find/connected query: is there a path connecting the two objects? Q. Is there a path connecting p and q? Union Find Data Type The goal is to design an efficient data structure for union-find with two commands, union and connected. It can be implemented by two different algorithms, Quick Find and Quick Union. Quick Find This quick find algorithm is called eager algorithm to solve dynamic connectivity problem. The data structure includes an Integer array id[] of length N and we consider p and q to be connected iff they have the same id. for (int i = 0; i < n; i++) id[i] = i; //set id of each object to itself Union To merge components containing p and q, change all entries whose id equals id[p] to id[q]. int pid = id[p]; int qid = id[p]; for (int i = 0; i < id.length; i++) if(id[i] == pid) id[i] = qid; Connected Check if p and q have the same id. return id[p] == id[q]; Java Implementation public class QF { public int[] id; public QF(int n) { id = new int[n]; for (int i = 0; i < n; i++) { id[i]=i; } } public void union (int p, int q) { int pid = id[p]; int qid = id[q]; for (int i = 0; i < id.length; i++) { if(id[i] == pid) { id[i] = qid; } } } public boolean connected (int p, int q) { return id[p] == id[q]; } } Quick Union It's an algorithm for solving the dynamic connectivity problem, also called "lazy approach". for (int i = 0; i < n; i++) parent[i] = i; //set parent of object at index i Root while (i != id[i]) i = id[i]; return i; Union To merge components containing p and q, set the id of p's root to the id of q's root. int i = root(p); int j = root(q); id[i] = j; Connected Check if p and q have same root. return root(p) == root(q); Java Implementation public class QuickUnionUF { private int[] id; public QuickUnionUF(int N) { id = new int[N]; for (int i = 0; i < N; i++) id[i] = i; } public int root(int i) { while (i != id[i]) i = id[i]; return i; } public boolean connected(int p, int q) { return root(p) == root(q); } public void union(int p, int q) { int i = root(p); int j = root(q); id[i] = j; } }

Integrating OpenCV with a Tkinter GUI in Python

K Akhilesh Saiju — Sun, 07 Jan 2024 11:52:59 GMT

OpenCV is a pretty handy tool while working with image processing and computer vision. It’s a large open-source library which can be accessed in a variety of programming languages, including Python. It has also been used for video analysis and real-world applications like CCTV footage analysis. OpenCV can be easily installed via the command console, to be used in various capacities while programming in Python. To use the various packages included in OpenCV, we can install it using pip, via the command, pip install opencv-python PIL (Python Imaging Library) is another useful open-source library which supports opening and using various image formats. By installing PIL as well, we can further improve our ability to integrate OpenCV packages in out programs. We can install PIL using the command: pip install Pillow A simple GUI utilizing tkinter is in general, more user-friendly and simpler to use. Hence, we can integrate OpenCV based programs with a tkinter GUI for easier access of end product. Tkinter can be installed with the command, pip install tk To demonstrate, we can create a Python program which utilizes an OpenCV package to open the user’s webcam. The user can then get the RGB codes of various regions in the captured feed via a tkinter GUI. To begin, we have to import the various packages to be used, from OpenCV, Tkinter and PIL import cv2 from tkinter import * from PIL import Image, ImageTk VideoCapture() is the function in package cv2 which can open a camera for video capturing, whereas the cvtColor() function converts the colourspace of the image provided. Here the first frame from the video feed is provided to be converted to RGB. vc = cv2.VideoCapture(0) cv2image= cv2.cvtColor(vc.read()[1],cv2.COLOR_BGR2RGB) In the GUI, buttons and labels are created to facilitate turning on the camera with user’s permission, and displaying the pixel data when any region in the video feed is clicked. window = Tk() window.title("Video feed") window.geometry("1200x700") button = Button(window,text="Click to open camera") button.grid(row=0, column=0) label = Label(window, height=500, width=650, text="Video feed here") label.grid(row=1, column=1, columnspan=1) label_pix1 = Label(window, height=5, width=30, text="Pixel Location: ") label_pix1.grid(row=2, column=0) label_pix2 = Label(window, height=5, width=30, text="Pixel colour scheme (RGB): ") label_pix2.grid(row=2, column=1) label_pix3 = Label(window, height=5, width=30, text="Pixel colour scheme (HexCode): ") label_pix3.grid(row=2, column=3) Two functions are created to implement these roles. Show_frames() displays the video feed from the webcam. At a set interval, a frame from the webcam feed is set as a PhotoImage object (from the PIL package) and updated in the GUI. def show_frames(): global cv2image cv2image = cv2.cvtColor(vc.read()[1],cv2.COLOR_BGR2RGB) img = Image.fromarray(cv2image) imgtk = ImageTk.PhotoImage(image = img) label.imgtk = imgtk label.configure(image=imgtk) label.after(20, show_frames) Pixel_colour() updates the information regarding the clicked pixel, every time a new pixel is clicked in the video feed. Also, the pixel data is printed onto the console. def pixelcolour(pix): print("Pixel location") print (pix.x, pix.y) label_pix1.config(text=f"Pixel Location: ({pix.x}, {pix.y})") print("RGB colour") print (cv2image[pix.x, pix.y]) label_pix2.config(text=f"Pixel colour scheme (RGB): {cv2image[pix.x, pix.y]}") print("Hex colour") bgr_list = cv2image[pix.x, pix.y].tolist() hex_str="#" for i in reversed(bgr_list): hex_str+=f"{hex(i)[2:]}" print(hex_str) label_pix3.config(text=f"Pixel colour scheme (RGB): {hex_str}") Setting the function to the button and the label in the GUI, and setting the mainloop for the window, button.config(command=show_frames) label.bind("", pixelcolour) window.mainloop() Running the program, a window is opened with a clickable button, which opens up the user's webcam. Click anywhere in the video feed et voila! Data about the selected pixel is displayed, which includes its position and colour scheme in RGB.

Cooler than KnapSack

Prakriti Bhattacharya — Fri, 05 Jan 2024 14:40:51 GMT

Familiarizing oneself with Design and Analysis of Algorithms this semester, one of the problems my professor keeps talking about is the KnapSack Problem. In the presented problem we have N items where each item has some profit and weight associated with it and also given a bag with capacity W, and profits associated with them being P. The task is to put the items into the bag such that the sum of profits associated with them is the maximum possible. The constraint here is we can either put an item completely into the bag or cannot put it at all. While the popular knapsack problem might deal with maximizing profit in a completely legal way, we borrow inspiration from data miners and contemplate robbing. Just that it is money instead of data and totally hypothetical of course. We consider that every house on a street has a specific amount of money. Now, our constraint here is that adjacent houses are no-goes. To minimize space complexity we keep tabs on the values of previous index prev and previous-to-previous index prev2and we can calculate the value for current index cur. class Solution { public: int rob(vector& A) { int prev2 = 0, prev = 0, cur = 0; for(auto i : A) { cur = max(prev, i + prev2); //here we compare the prev value and the sum of the alternate houses prev2 = prev; prev = cur; } return cur; } }; Now, if we were to add an additional constraint of the first and the last house being adjacent to each other - hence creating more of a cyclic system - we make further changes. public: int rob(vectornums) { int n=nums.size(); if (n<2) return n ? nums[0]:0; return max(robber (nums, 0,n-2), robber (nums,1,n-1)); } private: int robber(vectornums, int l, int r) { int prev2 = 0, prev = 0, cur = 0; for (auto i : A) { int cur = max(prev, i + prev2) prev2 = prev; prev = cur; } return cur; } Here, we divide the problem into parts and compute the maximum separately. Once by including the first house and excluding the last house, and again by excluding the first house and including the last house and finding its maximum.

RESTful APIs

Aryan — Fri, 29 Dec 2023 13:43:20 GMT

The concept of APIs is pretty fascinating when starting to code, because it allows us to integrate stuff into our projects which we would not want to develop ourselves. An API in short is just an interface for software applications to communicate. Its like the WhatsApp for applications. Let us assume you have an e-commerce application and you want to enable payments you can simply use the RazorPay API or PayTM API. Now lets dive deeper into how these APIs work, how to integrate them into out applications and by the end of this article you would have the working knowledge on how to create your own RESTful API. What is a RESTful API? A REST API is a way for computer programs to talk to each other over the internet. It follows certain rules to perform actions like getting data, adding new information, updating, or deleting. This is done typically using HTTP requests. There are multiple HTTP requests but we shall discuss GET, POST, PUT & DELETE as these are the ones which are most commonly used. Now recently I was brushing up my matplotlib and wanted to plot 4 graphs in real-time and all 4 of them would be running on separate threads. For this i wanted data which would keep changing every second. For this I used the 'Where the ISS at?' API. This API provides you with the following information about the ISS(International Space Station) every second. { "name": "iss", "id": 25544, "latitude": 50.11496269845, "longitude": 118.07900427317, "altitude": 408.05526028199, "velocity": 27635.971970874, "visibility": "daylight", "footprint": 4446.1877699772, "timestamp": 1364069476, "daynum": 2456375.3411574, "solar_lat": 1.3327003598631, "solar_lon": 238.78610691196, "units": "kilometers" } This API allows you to only GET the data, however this seems like a pretty basic example which showcases how these kinds of APIs work. Where are RESTful APIs used? In simple words a RESTful API allows CRUD based operations on a database. CRUD stands for CREATE, RETRIEVE, UPDATE, DELETE. Assume you have a Todo Application, in it you would be required to perform CRUD operations on the Todos. For this we can setup a RESTful API using which our web application or mobile application can send HTTP requests to perform the operations. How do you send HTTP Requests? To send HTTP requests, you typically use a programming language or a tool that supports HTTP communication. Here's a brief overview: If you are using python you are required to import the requests library. import requests response = requests.get("__endpoint_of_the_api_you_are_trying_to_hit__") # the url is also called the endpoint of the API print(response.body) If you are using javascript you can directly use the fetch function. fetch("__endpoint_of_the_api_youre_trying_to_hit__") .then(response => response.json()) .then(data => console.log(data)) .catch(error => console.error('Error:', error)); If you don't know what json is, think of it as something which serves as a common data format for exchanging structured information between web servers and clients. Apart from json XML is also used for similar purposes. How to create a RESTful API in python? We will be using the Django framework to create a REST API in python. Let us go over the step by step approach on how to do this. Install Django and initializing the project Install django using the pip install django command in the terminal. It is generally advised to create a virtual environment first and then install all the dependancies. After installing django we must create a project using the django-admin startproject myProject . You can change the name of the project to whatever you like. Now we must run the command pip install djangorestframework , it allows us to seamlessly create a REST API. Navigate to the settings.py file in your project directory, in our case it is myProject. Inside it add rest_framework to the list of installed apps like so. # myProject/settings.py INSTALLED_APPS = [ # other installed apps # add the line below to your code 'rest_framework', ] Now create an app using the command python manage.py startapp api. Creating a Model for the Data We will now create a model of the data we are trying to store. Let us make a model for a Todo App. # api/models.py from django.db import models class Todo(models.Model): title = models.CharField(max_length=50) desc = models.CharField(max_length=200) isDone = models.BooleanField(default=False) date = models.DateTimeField(auto_now_add=True) def __str__(self): return self.title We will also have to migrate this model to the database which by default is sqlite in django. You can change it to use a different one if you like. Run python manage.py makemigrations and then python manage.py migrate to do so. If you later make changes to the models by updating existing models or by adding new models just run python manage.py migrate after making changes. Creating a serializer In Django, a serializer is essential for converting complex data types, such as querysets or model instances, into Python data types that can be easily rendered into JSON for API responses. Create a serializers.py file in the api folder and change its content to match the following. # api/serializers.py from rest_framework import serializers from .models import Todo class TodoSerializer(serializers.ModelSerializer): class Meta: model = Todo fields = '__all__' Creating Views for Get, Create, Update, and Delete We will create two views: TodoGetCreate for listing and creating Todo items, and TodoUpdateDelete for retrieving, updating, and deleting individual Todo items. This specifies the model (Todo) and serializer (TodoSerializer) to use for these views, streamlining the implementation of CRUD (Create, Read, Update, Delete) operations for the Todo API. # api/views.py from django.shortcuts import render from rest_framework import generics from .models import Todo from .serializers import TodoSerializer class TodoGetCreate(generics.ListCreateAPIView): queryset = Todo.objects.all() serializer_class = TodoSerializer class TodoUpdateDelete(generics.RetrieveUpdateDestroyAPIView): queryset = Todo.objects.all() serializer_class = TodoSerializer Creating urls for the Views We will create two paths: TodoGetCreate for listing and creating Todo items, and TodoUpdateDelete for handling individual Todo items' retrieval, updating, and deletion based on their primary key. # api/urls.py from django.urls import path from .views import TodoGetCreate, TodoUpdateDelete urlpatterns = [ path('', TodoGetCreate.as_view()), path('', TodoUpdateDelete.as_view()), ] Note doing just this won't work as the paths for the project are being used from the urls.pywhich lies in the myProject directory. We will have to include the api.urls in the main urls.py. # myProject/urls.py from django.contrib import admin from django.urls import include, path urlpatterns = [ path('admin/', admin.site.urls), path('', include('api.urls')), ] Running the application Run python manage.py runserver to test it out. You should be able to see it working at http://127.0.0.1:8000/ .

Bash Scripting

Srivatsa — Fri, 22 Dec 2023 16:10:46 GMT

BASH stands for Bourne Again Shell. It is based on bourne shell and is mostly compatible with it features. Shells act as an intermediary between users and their operating systems, interpreting commands. Shells allow batch command execution and interactive command input, while they are not necessary for program execution. Consider a shell as a way to interact with your system, enabling you to carry out commands, launch programs, and automate difficult processes. The well-known shell BASH is merely a tool for running scripts and commands; it has nothing to do with managing files or configuring an operating system. In its interactive mode, BASH is frequently linked to command lines and prompts. But when scripts are executed, it can also function in a non-interactive manner. Files with lists of commands that can be used to automate tasks are called scripts. These commands are often run in a consecutive manner. Learn the fundamentals of interactive shells first; you can then combine them in scripts. A script is basically a sequence of commands in a file. Bash reads the file and processes the commands in order. It moves on to the next command only when the current one has ended. At first #!/bin/bash is added this header is called an interpreter directive (hashbang, shebang). It specifies that /bin/bash is to be used as the interpreter when the file is used as the executable in a command. Declaring some values to variable name: name="Kanhaiya" To call out the name 'Kanhaiya' we need to use the function echo: echo $name #output->Kanhaiya echo "hi my name is $name" #output->hi my name is Kanhaiya echo 'hi my name is $name' #output->hi my name is $name Editing the name: echo “${name/K/k}” #output->kanhaiya echo “${name:0:2}” #output->ka echo “${name,}” #output->ram (1st letter lowercase) echo “${name,,}” #output->ram (all lowercase) echo “${name^}” #output->Kanhaiya (1st letter uppercase) echo “${name^^}” #output->KANHAIYA (all uppercase) File conditions In Bash, file conditions are checks or tests that you can perform on files or directories. Some common file conditions include [[-e file]] #exists [[-r file]] #readable [[-h file]] #symlink [[-d file]] #directory [[-w file]] #writable [[-s file]] #size is >0 bytes [[-f file]] #file [[-x file]] #executable Chaining multiple commands or tests using && or || function comand1 || function comand2 if [condition1] && [condition2]; then #execute if true elif [condition3] || [condition4]; then #execute if true else #execute if all above false fi Arrays Arrays are used to store multiple pieces of data in one variable, which can then be extracted by using an index. Most commonly notated as var[index_position]. Arrays use indexing meaning that each item in an array stands for a number. In the array ['car', 'train', 'bike', 'bus'] each item has a corresponding index. All indexes start at position 0. Fruits=('Apple' 'Banana' 'Orange') Fruits[0]="Apple" Fruits[1]="Banana" Fruits[2]="Orange" echo "${Fruits[0]}" # Element #0 echo "${Fruits[-1]}" # Last element echo "${Fruits[@]}" # All elements, space-separated echo "${#Fruits[@]}" # Number of elements echo "${#Fruits}" # String length of the 1st element echo "${#Fruits[3]}" # String length of the Nth element echo "${Fruits[@]:3:2}" # Range (from position 3, length 2) echo "${!Fruits[@]}" # Keys of all elements, space-separated Fruits=("${Fruits[@]}" "Watermelon") # Push Fruits+=('Watermelon') # Also Push Fruits=( "${Fruits[@]/Ap*/}" ) # Remove by regex match unset Fruits[2] # Remove one item Fruits=("${Fruits[@]}") # Duplicate Fruits=("${Fruits[@]}" "${Veggies[@]}") # Concatenate lines=(`cat "logfile"`) # Read from file

Setting up your development environment for Node.js, Express.js & MySQL

Aryan — Thu, 21 Dec 2023 06:06:40 GMT

In this article, we will guide you through the crucial initial steps of setting up your development environment specifically tailored for Node.js, Express.js, and MySQL. By the end of this tutorial, you'll have a well-configured environment, providing you with the necessary tools and resources to kickstart your MERN stack learning experience. Let's pave the way for a seamless and efficient development process as you delve into the dynamic world of web development. If you are following the MERN stack which includes MongoDB, you can skip the MySQL part. Setting up MySQL Install the latest version of MySQL from their official website - https://dev.mysql.com/downloads/installer/ Make sure that you install the non-web version. In the setup select custom and proceed to install the MySQL Server, MySQL Shell, and MySQL workbench. Once installed setup the root password and do not change any other settings. Now once the installation is complete go to the folder where \bin is located where MySQL Server is installed. In my case the path looked like this - C:\Program Files\MySQL\MySQL Server 8.0\bin But this may change based on where you have installed it. Add a new path in your Environment Variables and paste the path there. The setup for MySQL is now complete ad we can verify this by entering MySQL --version in the command prompt. If installed correctly you should be able to see the installed version of MySQL. Setting up Node.js Install the latest stable version of node from their official website - https://nodejs.org/en/download/current While installing make sure to check the 'Automatically install the necessary tools.' checkbox. The setup for node is now complete ad we can verify this by entering node --version in the command prompt. If installed correctly you should be able to see the installed version of node. Installing Express.js Installing express is probably the easiest out of the lot. Open cmd and enter npm install express.

What is Bogo Sort?

Aryan — Tue, 19 Dec 2023 04:19:16 GMT

Sorting algorithms are precision-driven. Among all the effective and efficient sorting algorithms exists a highly inefficient and unconventional sorting algorithm - Bogo Sort. Instead of relying on strategic computation it relies on a game of chance. The pseudocode for bogo sort is given below: function isSorted(arr): for i from 0 to length(arr) - 2: if arr[i] > arr[i+1]: return false return true function bogoSort(arr): while not isSorted(arr): shuffleElements(arr) Bogo Sort is one of the algorithms which have the worst case time complexity of Ο(∞). This is because this technique relies on the sorted array to appear magically after a random shuffle of its elements. This is like playing the game of LUDO and hoping to roll a 6 on the die just wayy more intense. The best case scenario of this sort also like other sorting techniques occurs when the given input is already sorted. This algorithm performs best when the work for it has been already done. i.e. when it is not needed.🤦🏻

What are SQL Injections?

Aryan — Sat, 09 Dec 2023 22:48:38 GMT

In computing, SQL injection is a code injection technique used to attack data-driven applications, in which malicious SQL statements are inserted into an entry field for execution. This is the wikipedia definition. Let us try to further understand what it means. When using a SQL database with node.js and express.js we write queries to fetch the data from the database. A normal SQL query looks like: USE application; SELECT * FROM users; Where application is the schema and users is the table. A schema may consist of multiple tables. This query returns all rows in the users table. Now it may not always be useful for us to retrieve all the rows from the database. Let us say we want the row pertaining to the user with a specific user id. Instead of retrieving all the entries from the table and then iterating over them all till we find the desired entry we can write a query telling the database which entry we want and it will retreive only that. USE application; SELECT * FROM users WHERE user_id = '27129'; This is what the basic query structure looks like. Let us see how we can write a function in node.js to execute the query. const mysql = require('mysql2'); const pool = mysql.createPool({ host: process.env.MYSQL_HOST, user: process.env.MYSQL_USER, password: process.env.MYSQL_PASSWORD, database: process.env.MYSQL_DATABASE }).promise() async function getUser(user_id) { const [user] = await pool.query( "SELECT * FROM users WHERE user_id = " + user_id ); return user; } First we create a conncection to the MYSQL database and then we write an asynchronous function to get the user details. Looks pretty straightforward right. We are just taking the user_id as an input and passing it to the query directly. This user_id will mostly be coming from the frontend of our application in some way. The function will simply return the row pertaining to the user_id sent by the user. But what happens when a user_id which does not exist is entered? Well in that case nothing will be returned. So is our database secure? No. The variable user_id is passed to the SQL query in the form of an executable code and not pure data, which allows the user to write logic which may allow for some interesting scenarios. Let us see how this works. What does a computer return when given to check '1==1'? It return true. We will be using this simple information to write a very basic SQL injection. var user_id = ' OR 1=1'; Now when the function is executed with the following user_id the database will return all entries from the database where the user_id is ' ' or 1=1. Which unfortunately for us will be all entries😭. This means the user will be returned with all the data from our users table which might contain sensitive information such as names, emails, passwords, etc. So the 1=1 logic superceeds our logic of matching the entry where the user_id is given. So how do we handle this? How do we let the variable passed to the query be allowed to read as only data and not executable code. SQL has a very neat way of handling this. async function getUser(user_id) { const [user] = await pool.query( "SELECT * FROM users WHERE user_id = ?", user_id ); return user; }

Parallelism and Concurrency

Aryan — Mon, 06 Nov 2023 19:21:37 GMT

Concurrency and Parallelism are both concepts which are used in multithreading. Concurrency refers to the sequential completion of small pieces of multiple tasks such that over a period of time both of them are completed. This means that at any instant ony one task is being processed or executed. The execution of these tasks is achieved by breaking them down into smalled sub-tasks. The core processing the tasks quickly switches between tasks to make it appear that the tasks are being processed simultaneously however the process is entirely sequential. In the above given example we can see that both the tasks are broken down into smaller sub-tasks. These sub-tasks are performed such that both the tasks are completed almost together. The key point to remember here is that both the tasks start almost together and end almost together however at any point of time only one of them is being executed. Before understanding parallelism first let us understand what is parallel execution. Parallel execution means true simultaneous execution of tasks. This usually requires separate processing units or COREs as both tasks would be processed independant of each other on separate COREs. Parallelism is a process in which a task is divided into smaller sub-tasks and these sub-tasks are executed parallel. This means the task is executed much faster however requires multiple COREs or threads to run on. This means to save on time we must provide more computational power or in order to save on computational power we must compromise on the speed.

Executor Service in Java

Aryan — Sat, 04 Nov 2023 04:40:42 GMT

In the previous article we saw how we can implement basic multithreading in java. However to design efficient and reliable systems using multithreading we need to understand concepts pertaining to the topic of concurrency to actually build systems which can benefit from multithreading. Multithreading at its core allows us to execute tasks simultaneously with the help of multiple threads working at the same time. So if we want to execute a lot of tasks we should be creating as many threads as possible to finish the tasks quickly right? No. Creating threads is also an expensive process for the machine to do. We need to find the right balance between having enough threads so that our tasks get executed quickly and the number of threads we are using should be efficient for that system. ExecutorService is a way for us to manage and execute concurrent tasks by using threads, making it simpler to create and control the thread execution process. Let us look at how we can achieve this. Imagine the example of the taxis and passengers in the previous article. If we wish to drop all the arriving passengers to their hotels quicker we would have to hire more taxi drivers. Hiring too many taxi drivers is obviously an expensive task. In this if we were to hire an optimal amount of taxi drivers and a manager to manage the scheduling of the rides of the customers and handling the waiting ones then this process can be made faster and efficient in an resourceful manner. This is how a thread pool works. ExecutorService allows us to create Thread Pools. A thread pool is a group of threads which are managed by the ExecutorService. New tasks can be submitted for their execution to the service itself. Once the tasks are submitted they are queued up in a blocking queue. The blocking queue enqueues the tasks if all the threads in the thread pool are executing existing tasks and dequeues them once a thread gets free and the thread begins to execute the task which is dequeued. Note: To use the executor service you will have to import the concurrency package by java import java.util.concurrent; Fixed Thread Pool Now if we choose to use a fixed thread pool size we need to be mindful of the size. Usually it is recommended to use a pool size equivalent to or just under the available threads for that machine. A unique possibility is that if we are performing tasks which require the threads to be in a waiting state, then we can have more threads as otherwise there will be no threads to execute new tasks if all the threads are in waiting state and resources will not be used effectively. Cases like this can be observed while using asynchronous programming. ExecutorService myService = Executors.newFixedThreadPool(10); for (int i=0; i < 100; i++) { myService.execute(new Tasks()); } Cached Thread Pool A cached thread pool is very similar to a fixed thread pool just the blocking queue is replaced by a synchronous queue. A synchronous queue can hold only one task at a time. When it recieves a new task it looks for any free threads in the thread pool. If it finds a free thread then the task is executed by that particular thread otherwise a new thread is created to execute that task. Since this may lead to the creation of a large number of threads, the newly created threads are terminated if they are found to be idle for a specific amount of time to avoid having too many threads. ExecutorService myService = Executors.newCachedThreadPool(); Single Thread Executor A single thread executor is a thread pool with a single thread. It fetches new tasks and executes them from the blocking queue. If the thread is terminated in the process of execution of a task it is recreated. This way there always exists one thread in the thread pool. Handling Rejections When all the threads are busy the new tasks are stored in the blocking queue, but the blocking queue also must have its limitations right. What would happen to a new task if all the threads are busy and the blocking queue is also full. If new thread creation is not an option then the task is rejected by the thread pool, this rejection can happen in one of four ways - Abort Policy, Discard Policy, Caller-Runs Policy and Discard Oldest Policy. Let us see how each of these handles the rejection of tasks. Abort Policy: Simply throws a RejectedExecutionException. Discard Policy: Discards the task. Caller-Runs Policy: Executed the task on the caller's thread itlsef. Discard Oldest Policy: Discards the oldest task in the blocking queue and the new task is added to the queue. Shutdown Initiations We have seen how to start an ExecutorService and how it works, but how do we shut it down? Since the threads are executing the tasks wouldn't the shutdown affect the execution of those tasks? We can shutdown an ExecutorService using the following command. ExecutorService myService = Executors.newFixedThreadPool(15); for (int i=0; i<50; i++) { myService.execute(new Task()); } myService.shutdown(); When we use the command myService.shutdown(), it only initiates a shutdown. Which means that all the tasks being executed by the threads and all the tasks in the blocking queue will be completed first and then it will shutdown. In this duration if any new tasks are submitted to the thread pool a RejectionExecutionException will be thrown. List queuedTasks = myService.shutdownNow(); The above command is used when we want to shutdown immediately after the threads finish executing their current tasks. The tasks queued up in the blocking queue are returned as a list. The status of the shutdown can be checked with the two commands - isShutdown() and isTerminated(). isShutdown() is used to check if the shutdown process has been initiated, however there is no way to know if the entire service has been shutdown only using isShutdown(), for that we will have to use the isTerminated() function as it is used to check if all the tasks are completed and hence the service has been shutdown. // returns true if the shutdown has been initiated. myService.isShutdown(); // returns true if all the tasks are completed, including the queues ones. myService.isTerminated();

Synchronization in Java

Aryan — Sun, 29 Oct 2023 06:33:28 GMT

Synchronization allows for only one thread to access a piece of code or an object at any given moment. This prevents simultaneous requests to access the data by multiple threads. It is a mechanism put in place to control access to te resources which are shared among all the threads. This is done to ensure accuracy of the processes being carried out by threads and so that they dont interfere with each other or also otherwise known as interleaving, all this can lead to the data being corrupted. Let us try to understand this using an example. Consider there exists a library a the TeckKnowHow Club. Members of the club can borrow books which they find interesting and return them once they have finished reading them. Now considering this library is multithreaded and different threads handle the borrowing and returning of books by various members of the club, it is possible that two threads try to borrow the same book for different users. This may lead to inconsistencies in the book status and also not make for a very reliable system. In such a case synchronization needs to be introduced to tackle this issue. In simple words synchronization is used to avoid thread interference and concurrency issues. Thread intereference: When two or more threads try to read/write to/from a single or multiple shared resources, it may lead to unpredictable and incorrect results. Concurrency issues: When multiple threads execute concurrently and their order of execution influences the result/outcome. A few basic concurrency issues are listed below : Race Condition When multiple threads concurrently access the same shared resource, the outcome depends on the order of execution of the threads, hence giving us inconsistent outputs. Deadlock When two or more threads are waiting for each other to release a shared resource, may lead to them being unable to proceed. Starvation When multiple threads are waiting for each other to release the resource, it may lead to a few threads being unable to access the resource entirely The process of gaining exclusive access over a shared resource by a particular thread is called Locking. The lock on the shared resource exists till the process being executed by the thread is completed. When a thread acquires a lock, it means that it has exclusive permission to execute the code within the locked region. So how can you implement this concept of synchronization in code? There are two ways in which this can be done: Using synchronization blocks. Using synchronized methods. Synchromization Blocks You can write certiain pieces of code inside a synchronization block, which will synchronize it for you. This method allows for more detailed control over synchronization. This is due to the synchronization being applied only to parts of code and not entire functions or methods. synchronized (object) { // here object refers to the object for which the shared resources will be locked. // code which needs to be synchronized. } Synchronized Methods An entire method can be declared synchronized by using the 'synchronized' keyword in the mthod declaration. This is used when we want the entire functionality of a method to be synchronized. synchronized function name (args) { // functionality of the method. }

Multithreading in Java

Aryan — Tue, 24 Oct 2023 21:35:19 GMT

Multithreading is a way for us to execute the same process across multiple threads. A thread can be understood as a small unit which executes a program/process. A process can be executed by multiple threads at the same time simultaneously, this is known as multithreading. These threads can share the same resources like memory, but they can also have their own independant resources. Threads have their own program counters, stack and even their own registers. This allows for independant processing across threads, which means that the output of a single thread is independant of other threads. This means that even if a particular thread runs into a problem or exception the processing of other threads will not be affected by it. This is a major advantage when it comes to multithreading. Let us understand this with a further example : let us assume there is a single taxi standing outside a busy airport. Assuming that the passengers arriving use only taxis to commute to their hotels, the taxi driver will have to make successive trips to drop all the passengers to their respective hotels. This will be a very time consuming process and labour intensive on the taxi driver as he will have to drop all the passengers one after the other. This problem can be fixed by having multiple taxis available. This approach divides the load and makes the process much faster also without tiring out the drivers too much. Multithreading therefore allows us to run several threads in parallel, taking advantage of multi-core processors to perform tasks concurrently. Let us try to understand how multithreading can be implemented in java. There are two ways in which this can be achieved : Extending Thread in the class where multithreading is performed. Implementing the Runnable method. Extending Thread First we create a class and extend Thread as te name suggests, inside it we will override the run() method. Inside the run() method you should put the code for the process that you want a single thread to or each thread to perform. This can be as simple as counting from 1 to 5 or as complex as simultaneously running different data points through a machine learning model. Make sure to use a try-catch block inside the run method to handle the exceptions. Once we have defined the task for each thread we will create an object of the class in our main method and use the start() method to begin the execution of a new thread. When the start() method is called the JVM(java virtual machine) creates a new thread and invokes its run() method. So if you want 10 threads to perform the mentioned task then you should create 10 different objects and call the start() method for each of them. This can be done simply using a for loop. package Multithreading; import java.util.Date; class MultithreadingClass extends Thread { @Override public void run() { try{ for(int i = 0; i < 5; i++){ Date d = new Date(); System.out.println("Thread " + Thread.currentThread().threadId() + " is running at : " + d); Thread.sleep(1000); } } catch (Exception e){ System.out.println("Exception is caught : " + e); } } } public class DateTime { public static void main(String[] args) { int n = 5; for (int i = 0; i < n; i++) { MultithreadingClass object = new MultithreadingClass(); object.start(); } } } We can visualise the working of this in real-time even better if we use the command 'Thread.sleep(1000)' immediately after it performs the task in the run() method. This stops the processing of each thread for exactly 1000 milliseconds or 1 second after completing their single task. Implementing Runnable Since we are not extending Thread anymore we cannot call the start() method directly in our main method. Instead we will create a new Thread in the main method and pass in the object of the class that implements the Runnable Interface. Now we can call the start() method for the Thread. package Multithreading; import java.util.Date; class MultithreadingClass implements Runnable { @Override public void run() { try{ for(int i = 0; i < 5; i++){ Date d = new Date(); System.out.println("Thread " + Thread.currentThread().threadId() + " is running at : " + d); Thread.sleep(1000); } } catch (Exception e){ System.out.println("Exception is caught : " + e); } } } public class DateTime { public static void main(String[] args) { int n = 5; for (int i = 0; i < n; i++) { MultithreadingClass object = new MultithreadingClass(); Thread T = new Thread(object); T.start(); } } } The output for the above code looks like this From this we can clearly see that 5 threads are working simultaneously, i.e. Threads - 29, 30, 31, 32, and 33. Note that in each cycle the threads are not printing in the same order, which means some threads finish the same task faster than others and it cannot be predicted which thread will be faster as it is random and it depends on the systems resource allocation.

Removing nth Node From End of Linked List

Aryan — Tue, 24 Oct 2023 17:39:31 GMT

The most basic approach to this problem would be to iterate over the entire linked list once to find out the length of the list. Then iterate over it again to locate the node to be removed. This can be done in the following way: Let n be the length of the list and we need to remove the mth element from the end so we will have to remove the (n-m)th element from the list. However this approach requires having to iterate over the list twice and hence the time complexity will be O(2n). We can solve this problem in O(n) time complexity also by using the following approach: Declare 2 pointers - last and prev and initialize both of them to to point to head Iterate the last pointer m times. Iterate both the pointers last and prev till last.next == null. After following these 3 steps out prev pointer will be pointing to the node just before the one that needs to be removed from the list. Now we can simply remove the required node by pointing prev.next to prev.next.next static void remove_nth_node(LinkedList l, int n) { Node last = l.head; Node prev = l.head; for (int i = 0; i < n; i++) { last = last.next; } while (last.next != null) { last = last.next; prev = prev.next; } prev.next = prev.next.next; }

Reversing a Linked List

Aryan — Tue, 24 Oct 2023 16:46:04 GMT

If we had to reverse the elements of an array we would have to iterate over then entire array and swap the ith element with the (n-i-1)th element where n is the size of the array. Now in a linked list we are not required to swap the data in the nodes. Instead we can achieve the same by just reversing the pointers, initializing the head to the last element of the given linked list and point the head of the original linked list to null. So we will create 2 variables; one of them will always point to the node that the current node needs to point at, and the other will always point to the node the current node is currently pointing at. If we iterate till the current node - starting from head is not 'null' then we would have reversed the entire linked list. static void reverse_linked_list(LinkedList l) { Node prev = null; Node temp = l.head; while (l.head != null) { temp = l.head.next; l.head.next = prev; prev = l.head; l.head = temp; } l.head = prev; }

Linked Lists

Aryan — Tue, 24 Oct 2023 16:27:53 GMT

Linked lists are linear data structures, which consist of nodes. These nodes store data in them as well as point to the next node in the list. So each node in the list has 2 attributes, one of the datatype you are trying to store which will hold the actual data and one of the datatype 'Node' which will point to the next node in the list. Linked lists are better than arrays as we do not have to move around as many elements in the list when doing operations such as insertions, deletions and updations at the beginning, end or at a random node in the list as compared to arrays. This makes using them more efficient. I will be posting solutions of questions solved in java based on the comcepts of linked lists.

Snake Game in Java

Aryan — Wed, 04 Oct 2023 11:48:22 GMT

In my third semester at university, I had taken the 'Computer Programming in Java', while learning all the core concepts of java I thought it would be fun to try and make a game in the process. I tried making the snake game! The logic of the game is very simple and straight forward. If the snake head hits its own body or the edges of the screen then game over, if the snake head hits the apple then increment the length of the snake by 1 and increment the score also by 1, allow turning only in 90 degrees to avoid turning the snake head into itself - say for instance the snake is moving forward and the user presses the down arrow key; we would not want the snake to make a 180 degree turn as that will lead to a game over. I learnt a lot of new stuff while making this game like - how to build a basic graphic user interface in java using the inbuilt packages, and how to use Key Adapters : they were used in making the snake turn when the arrow keys were pressed. You can find the source code to the game here - https://github.com/AryanBhirud/SnakeGame_in_Java/

Determinant of matrices using Python

Madhur — Thu, 27 Oct 2022 16:54:24 GMT

In this article, I will teach you to find the determinant of a matrix using the Python programming language. Prerequisites: In programming (Python) - Lists and operations on lists Concept and understanding of Recursion In Maths - A basic understanding about Matrices Finding a Determinant of a matrix Revision: A list in Python is a sequence of objects and is denoted by square brackets. my_list = [1, 5, 3, 7] Recursion is a concept of calling the same function inside the function itself. def factorial(n): if n == 1: return n else: return n * factorial(n-1) A matrix is an arrangement of numbers in a specific, systematic order of rows and columns, like this - A determinant of a matrix is an operation on a matrix which returns a single number, and is often denoted like : Determinant of a matrix A = |A| A determinant of a matrix is calculated as follows- = 3 × (2×6 - 4×3) - 4 × (2×6 - 3×5) + 7 × (2×4 - 5×2) = 3 × 0 - 4 × (-3) + 7 × (-2) = 0 + 12 - 14 = -2 Solution: We will design a function that takes in a 2d list as input and gives a single number as the output. The input 2d-list would be our matrix and the output would be the determinant, or the final answer. We can find the determinant of the matrix using the standard technique (Laplace expansion)- So, we will take the first row of the matrix element-by-element and multiply it with the determinant of the remaining matrix, like, a11 × |a22 a23 ...| ⠀ |a32 a33 ...| ⠀ |... ... | Thus, we will use the technique of recursion in this way: def determinant(matrix): ans = 0 for my_element in first_row_of_matrix: ans = ans + (-1)^index_of_my_element * my_element * determinant(matrix_without_the_row_and_column_of_my_element) return ans But we need to define the end-condition for our recursion somewhere. So, the best and the simplest way is when the input matrix is just a 2×2 matrix, and we can just cross-multiply its elements. So, our code becomes - def determinant(matrix): if size(matrix) == (2×2): return cross_multiplication_of_elements_in_matrix else: ans = 0 for my_element in first_row_of_matrix: ans = ans + (-1)^index_of_my_element * my_element * determinant(matrix_without_the_row_and_column_of_my_element) return ans That's it! This is the whole logic, and now we just have code this in python! The first 'if' statement is simple to write - def determinant(matrix): if len(matrix) == 2: return matrix[0][0] * matrix[1][1] - matrix[0][1] * matrix[1][0] In the else statement, we first have to slice the matrix so that the row and the column of the selected element are not present in the sliced matrix. A matrix 'matrix2' is initialized which is the sliced version of the input matrix. def determinant(matrix): if len(matrix) == 2: return matrix[0][0] * matrix[1][1] - matrix[0][1] * matrix[1][0] else: ans = 0 for element in range(len(matrix)): matrix2 = [] # This part of the code slices matrix to matrix2 for count in range(len(matrix)-1): matrix3 = [] for count2 in range(len(matrix)-1): matrix3.append(0) matrix2.append(matrix3) # Now, matrix2 is a blank matrix of filled with zeros of the required size (size when one row and one column is sliced from original matrix) # Now, we start 'filling' matrix2 with the required elements from input matrix i = 0 while i < len(matrix)-1: j = 0 a = 0 while j < len(matrix)-1: if j == element: a += 1 matrix2[i][j] = matrix[i+1][j+a] j += 1 i += 1 # matrix2 is created, we just now calculate the determinant and add it to the ans ans += ((-1)**elementx) * matrix[0][element] * (determinant(matrix2)) return ans Footnotes: The code written by us is in no way fast - there are many optimizations available. This code is written from the perspective of understanding and not from the perspective of speed. Thank you for taking out time to read this. I hope you learnt something new!

C++ Interactive Programs using CGI

Aryan — Thu, 20 Oct 2022 16:28:18 GMT

C++ Interactive Programs using CGI

Web based Loan Amortization Calculator using C++ CGI

Aryan — Thu, 20 Oct 2022 16:25:59 GMT

Web based Loan Amortization Calculator using C++ CGI

C++ Loan Amortization Calculator

Aryan — Thu, 20 Oct 2022 16:24:12 GMT

C++ Loan Amortization Calculator

Deploying C++ CGI Application on Microsoft IIS

Aryan — Thu, 20 Oct 2022 15:34:37 GMT

Deploying C++ CGI Application on Microsoft IIS

Running Visual Studio Code & GNU C++ on Ubuntu for

Aryan — Thu, 20 Oct 2022 15:27:55 GMT

Running Visual Studio Code & GNU C++ on Ubuntu for

Install & Configure C++ Package / Library Manager - VCPKG

Aryan — Thu, 20 Oct 2022 15:22:15 GMT

Install & Configure C++ Package / Library Manager - VCPKG

Hello world!

Aryan — Tue, 11 Oct 2022 13:29:00 GMT

Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

Configuring Ubuntu 22.04 LTS Virtual Machine on Windows 11 Hyper-V

Aryan — Tue, 30 Oct 2018 18:02:08 GMT

Ubuntu is a Linux distribution based on Debianand composed mostly of free and open-source software. Ubuntu is officially released in three editions: Desktop, Server, and Core for Internet of things devices and robots. All the editions can run on the computer alone, or in a virtual machine.' Ubuntu is released every six months, with long-term support (LTS) releases every two years. As of 21 April 2022, the most recent long-term support release is 22.04 ("Jammy Jellyfish"). Configuring Ubuntu Virtual Machines on Windows 11 Hyper-V is very easy and quick. The Hyper-V offers Virtual Machine Gallery from where we can download the VM. This requires a good internet connection as the VM size is approx. 2.4-2/5GB. Step 1: Invoke Hyper-V Manager Login to Windows 11 as Administrator user. A normal user cannot see ‘administrative tools’ menu.You need to search for the ‘Hyper-V Manager’ MMC Snippet and “Run As Administrator” by right clicking it. You will see the Hyper-V Manager as below. Step 2: Select Hyper V Node & Click "Quick Create" On the left hand side Tree Pane, select the Hyper V Node. Right Click Step 3: Virtual Machine Gallery Dialog will be opened Select the "Ubuntu 22.04TLS" from the List. Here, you can click on "More Options" to choose Virtual Machine Settings and the Name of the "Virtual Machine". You can change the settings later also. Click "Create Virtual Machine" will create a Virtual Machine and Download the Image from Ubuntu. If you have a good internet connection this will be quicker. Step 4: The Virtual Machine is ready You will see the below dialog Step 5: Some important points - weird observations The Newly Created Virtual Machine from Gallery - the Wizard does not offer us to choose no of Processors and "By-Default" assigns HALF of the Cores on your node. Select Virtual Machine, and Right Click Settings: and review all hardware configuration. I had 20 cores to my PC and hence the QuickCreate assigned 10 Core to the new VM. You may want to change that. Step 6: Review Boot Sequence once Ensure that the HDD is selected as first Boot Sequence Step 7: Start & Connect the Virtual Machine Follow the On-Screen Process to setup Ubuntu. Configure your PC Name and Root User Password Note- Keep the Network Switch as - Default Network Switch so that the newly created Virtual Machine has access to Internet and you can download and install tools like IFCONFIG as NETTOOLS are not included by-default. You can open "Terminal" and Run following command sudo apt-get install -y net-tools

Installing Windows 11 Virtual Machine in Hyper-V

Aryan — Mon, 29 Oct 2018 18:02:08 GMT

Windows 11 is the latest major release of Microsoft's Windows NToperating system, released in October 2021. It is a free upgrade to its predecessor, Windows 10 (2015), available for any Windows 10 devices that meet the new Windows 11 system requirements.The basic system requirements of Windows 11 differ significantly from Windows 10. Windows 11 only supports 64-bit systems such as those using an x86-64 or ARM64 processor; IA-32 processors are no longer supported.Step 1: Invoke Hyper-V ManagerLogin to Windows 11 as Administrator user.A normal user cannot see the ‘administrative tools’ menu.You need to search for the ‘Hyper-V Manager’ MMC Snippet and “Run As Administrator” by right clicking it.Step 2: Select Hyper V Node, Click “New”, then click on “Virtual Machine” Once you select virtual machine the “new virtual machine wizard” would open up like so. Click on next and then name your virtual machine. Generation 1 Virtual Machines support both 32-bit and 64-bit operating systems.However Windows 11 supports only 64-bit systems, so we would choose Generation 2.Note : The generation of the virtual machine cannot be changed later on. The default initial assigned startup memory would be 1024 MB, change that to 4096 MB, and click on next. By default the network adapter is set to ‘not connected’; change the setting to ‘Default Switch’. Here we have the option of connecting a virtual hard disk. If you have a pre-existing virtual hard disc you can choose the second option and specify its location. If you do not have a pre-existing virtual hard disc you can create a virtual hard disc. Simply specify its name, location and size. Select 'Install an operating system from a bootable image file' and select the path of the image copy of windows 11 which you might already have installed on your computer. Click on finish to complete the VM setup. Step 3: Changing the Settings of our Virtual Machine Now right-click the virtual machine and click on settings. By default only 1 virtual processor is allocated for the virtual machine. You may increase it to whatever suits your needs. My PC has 20 virtual processors, so I allocated 4 to this VM. Click on Apply; then in the menu on the left side click on Security. In the Encryption Support check the first checkbox. If this step is skipped, windows cannot be installed on your VM. You will get an error like this. Step 4: Windows 11 setup on Virtual Machine Right-click the VM and click on connect, as shown below. Once you click on connect, you will need to start your VM. Based on your preferences select the options and click next. Click on Install Now to install Windows 11. Enter your own Product Key and click on Next or skip this step entirely by clicking on I don't have a product key. Tick the checkbox and click Next. Once you click Next you will have 2 choices for the installation. One of them would be default and the other would be custom. Select custom for more control over the installation process. Click Next. Once the installation is complete, your virtual machine will restart. After restarting the windows 11 setup begins. Step 5: Setting up Windows 11 Select your country and click yes. Select the keyboard layout/preferred input method. Add a second keyboard layout if you want to, or you can skip this step entirely by clicking skip. Name your device. You don't need the same name for this and the virtual machine, I just did it for the sake of convenience. Choose the preferred option and proceed. Add your Microsoft account by signing in. After signing in to your Microsoft account, you will get the option to restore the files backed up over Onedrive, settings & preferences and your apps which you had on a previous device with your Microsoft account set up. Click on view more options. I selected the Set up as new device option, but you can choose other available options also. Set up a pin for this machine. Choose the privacy settings. I unchecked all of them, but you can uncheck as many or as little as you like. Here Windows allows you to backup the files on this virtual machine using OneDrive so that you can access them from other devices also. If you do not want to back up the files you can click Only save files to this PC. It will take a few minutes to get everything ready. Once windows 11 loads you can choose the size of the desktop by dragging the slider. And there you have it we have successfully, created a virtual machine on Hyper-V with Windows 11 on it!

Installing Hyper-V on Windows 11

Aryan — Fri, 26 Oct 2018 18:02:08 GMT

Installing Hyper-V on Windows 11

Installing & Configuring GNU C++ Complier on Windows 11

Aryan — Thu, 25 Oct 2018 18:02:08 GMT

Quisque dapibus, velit eget ullamcorper suscipit, sapien ligula hendrerit lectus, vitae tristique sapien velit ac lectus. Mauris ullamcorper nisi sit amet est vestibulum interdum. Integer venenatis rutrum ipsum. Pellentesque sollicitudin turpis eu nibh ornare, eu pretium risus pharetra. Est egestas, non facilisis nisl mattis. Nunc a fermentum tortor. Nulla vehicula nunc tortor, eu sagittis dui dictum a. Integer erat urna, gravida eu cursus non, posuere nec quam. 1. Nullam sit amet orci turpis. Cras congue mi sit amet ullamcorper dapibus. Vestibulum pulvinar rutrum odio, et mollis elit blandit sit amet. Suspendisse semper elit eu volutpat tincidunt ante augue sodales justo auctor mattis mi dui at elit duis vel porttitor tellus 2. Aenean vestibulum volutpat est ut vestibulum. Pellentesque mauris ante, elementum sed arcu id, tincidunt venenatis neque. Aliquam consectetur magna ut adipiscing molestie. Phasellus faucibus pretium odio. Ut ut lacus est. Nam consectetur nulla eget nibh auctor fermentum. Aenean feugiat semper arcu id suscipit enim fermentum et. Curabitur tellus neque, eleifend et turpis dictum, lacinia porta nibh. Nulla sed tincidunt felis. 3. Phasellus malesuada sed metus nec tristique Duis semper, eros nec aliquet lacinia, tortor erat interdum dui, non vestibulum massa odio et dui. Pellentesque in nibh libero. Sed at velit fermentum, porttitor nisi eu, condimentum orci. Sit amet risus cursus, nec lobortis magna aliquam. Fusce in mauris neque. Etiam vitae felis mauris. Vestibulum eu odio at velit varius hendrerit tempus vel libero. Vivamus ultricies sapien sed posuere scelerisque. Aliquam mattis eros et semper laoreet. Donec eget consequat sem Cras congue ac orci quis consequat. Sed purus libero, volutpat non facilisis et, adipiscing sed lacus. Vestibulum aliquam porta quam, vel varius tortor eleifend eu. Donec at erat et sem bibendum hendrerit Vestibulum turpis augue, rhoncus a laoreet sed, vestibulum a nunc. Integer id tempor elit, sed commodo est. Sed sollicitudin nibh faucibus, vulputate turpis at, imperdiet mi. Ut venenatis mauris vel consectetur convallis. Donec scelerisque at eros eu convallis. Nulla consequat ut eros et auctor. Fusce semper pretium massa sit amet mollis. Proin sit amet condimentum nulla, vitae cursus est. Sed venenatis nisi vitae turpis dictum sollicitudin. Nunc quis metus porttitor neque luctus ultricies.

Configuring GNU CGICC CGI Library on Windows 11

Aryan — Wed, 24 Oct 2018 18:02:08 GMT

Curabitur quis egestas odio, ac tincidunt neque. Cras egestas sapien eu egestas iaculis. Pellentesque et velit adipiscing, imperdiet urna vel, venenatis libero. Fusce semper tortor vel convallis blandit. Nullam vel tempor mi. Maecenas convallis leo tempor Est egestas, non facilisis nisl mattis. Nunc a fermentum tortor. Nulla vehicula nunc tortor, eu sagittis dui dictum a. Integer erat urna, gravida eu cursus non, posuere nec quam. Nullam sit amet orci turpis. Mauris egestas dictum porttitor. Cras congue mi sit amet ullamcorper dapibus. Vestibulum pulvinar rutrum odio, et mollis elit blandit sit amet. Suspendisse semper elit eu volutpat tincidunt ante augue sodales justo auctor mattis mi dui at elit duis vel porttitor tellus Aenean vestibulum volutpat est ut vestibulum. Pellentesque mauris ante, elementum sed arcu id, tincidunt venenatis neque. Aliquam consectetur magna ut adipiscing molestie. Phasellus faucibus pretium odio. Ut ut lacus est. Nam consectetur nulla eget nibh auctor fermentum. Aenean feugiat semper arcu, id suscipit enim fermentum et. Curabitur tellus neque, eleifend et turpis dictum, lacinia porta nibh. Nulla sed tincidunt felis. Phasellus malesuada sed metus nec tristique. Duis semper, eros nec aliquet lacinia, tortor erat interdum dui, non vestibulum massa odio et dui. Pellentesque in nibh libero. Sed at velit fermentum, porttitor nisi eu, condimentum orci. Aliquam vestibulum orci sit amet risus cursus, nec lobortis magna aliquam. Fusce in mauris neque. Etiam vitae felis mauris. Vestibulum eu odio at velit varius hendrerit tempus vel libero. Vivamus ultricies sapien sed posuere scelerisque. Aliquam mattis eros et semper laoreet. Donec eget consequat sem. Suspendisse sit amet varius justo. Cras congue ac orci quis consequat. Sed purus libero, volutpat non facilisis et, adipiscing sed lacus. Vestibulum aliquam porta quam, vel varius tortor eleifend eu. Donec at erat et sem bibendum hendrerit Vestibulum turpis augue, rhoncus a laoreet sed, vestibulum a nunc. Fusce semper pretium massa sit amet mollis. Proin sit amet condimentum nulla, vitae cursus est. Integer id tempor elit, sed commodo est. Sed sollicitudin nibh faucibus, vulputate turpis at, imperdiet mi. Ut venenatis mauris vel consectetur convallis. Donec scelerisque at eros eu convallis. Nulla consequat ut eros et auctor. Sed venenatis nisi vitae turpis dictum sollicitudin. Nunc quis metus porttitor neque luctus ultricies.

TechKnowHow Club

Serial Port for emulated esp32 using QEMU(?)

Virtualized sensors for esp32 emulator

esp32 on QEMU

BitFlash_Client - Open Source FOTA for Everyone

Object Detection - The Basics.

Setting Up HTTPS on IIS in 2024

Building CodeClip: Copy Your Entire Codebase Instantly

Seamless User Management: Integrating Firebase Authentication with Odoo

Captive Portals: How to automate them

Activation Functions in Neural Networks

Custom Object Detection using TensorFlow Lite

Automating MySQL Backups

Serving Node Applications on Microsoft IIS

A Hacky Way to Scrape Linkedin

Extract LinkedIn User Data

Voxa - Our Journey from Idea to Implementation

Object detection using OpenCV

Kubernetes: The only Intro you will need

Communicating via the SerialPort using Electron.js

Serial Reader

This data is being streamed from the serial port:

What is Bootstrap and how to implement its components.

Club TechKnowHow!

Welcome to Coder.com

The most trusted coding blog

Award winning coding blog

Featured post

Post title

Special title treatment

Git and GitHub - How to Setup and Get Started

Server-Side scripting with XAMPP

Welcome to Image of an Egg

Here's the aforementioned image

Welcome,

Here is your egg.

I don't like it Take me back to original egg

A Hands-on Approach to Docker

Creating a REST API using Node.js & Express.js

Dynamic Connectivity Problem

Integrating OpenCV with a Tkinter GUI in Python

Cooler than KnapSack

RESTful APIs

Bash Scripting

Setting up your development environment for Node.js, Express.js & MySQL

What is Bogo Sort?

What are SQL Injections?

Parallelism and Concurrency

Executor Service in Java

Synchronization in Java

Multithreading in Java

Removing nth Node From End of Linked List

Reversing a Linked List

Linked Lists

Snake Game in Java

Determinant of matrices using Python

C++ Interactive Programs using CGI

Web based Loan Amortization Calculator using C++ CGI

C++ Loan Amortization Calculator

Deploying C++ CGI Application on Microsoft IIS

Running Visual Studio Code & GNU C++ on Ubuntu for

Install & Configure C++ Package / Library Manager - VCPKG

Hello world!

Configuring Ubuntu 22.04 LTS Virtual Machine on Windows 11 Hyper-V

Installing Windows 11 Virtual Machine in Hyper-V

Installing Hyper-V on Windows 11

Installing & Configuring GNU C++ Complier on Windows 11

Configuring GNU CGICC CGI Library on Windows 11

I don't like it

Take me back to original egg