Ezio Ruan's Blog

—title: “Docker Best Practices: Writing Optimized and Secure Dockerfiles”date: 2022-10-05 11:20:00tags: Docker, Containerization, DevOps, Best Practices—Docker has revolutionized how we build, ship, and run applications. At the heart of a good container is a well-crafted Dockerfile. While it’s easy to get an application running in a container, creating an image that is small, secure, and fast requires following a set of best practices. This guide covers the essentials for writing optimized and secure Dockerfiles.

Large Language Models (LLMs) like GPT-4 are incredibly powerful, but they have two fundamental limitations: they don’t know anything about events that occurred after their training cut-off date, and they don’t have access to your private, domain-specific data. This can lead to outdated or generic responses. The solution to this problem is a powerful architectural pattern called Retrieval-Augmented Generation (RAG).

Why Use Fauxpilot?

Fauxpilot is an open-source alternative to GitHub Copilot, leveraging the power of large-scale models to deliver a range of AI-driven functionalities, including dialogue and code suggestions. In this article, we’ll focus exclusively on its code suggestion capabilities, with other features to be detailed in future posts.

Local Setup

To run Fauxpilot, the following prerequisites must be met:

• A Windows or Linux system with an NVIDIA graphics card.
• At least 50GB of local storage space (required for the large model).
• Docker and nvidia-container-toolkit.
• The curl and zstd command-line tools, necessary for model downloads.

I am using Ubuntu.

The Docker installation is simple and won’t be elaborated here. The NVIDIA Container Toolkit is a project that enables containers to utilize NVIDIA GPU resources. The following commands can be used for installation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# Configure the production repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Configure the repository to use experimental packages: 
sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update and install:
sudo apt-get update && apt-get install -y nvidia-container-toolkit

# Configure the container runtime to use NVIDIA GPUs:
sudo nvidia-ctk runtime configure --runtime=docker

Then, simply restart Docker:

1

sudo systemctl restart docker

Installation methods for other systems can be found in the complete installation guide.

Next, enter the project root directory and run the startup script directly:

1
2

cd fauxpilot
./setup.sh

The Python packaging ecosystem is powerful but notoriously fragmented. A typical project might use venv for environments, pip for installation, pip-tools for locking dependencies, and maybe pipx for running tools in isolation. This collection of tools, while functional, can be confusing for newcomers and cumbersome for experts. Enter uv, a new tool from Astral (the creators of the ruff linter) that aims to change everything.

The rise of Large Language Models (LLMs) has opened a new frontier for application development. However, building, deploying, and managing these AI-powered apps can be complex, involving everything from prompt engineering to backend service management. To address this, a new category of tools has emerged: LLM-Ops platforms. Among the most popular open-source options are Dify and FastGPT, which aim to simplify the entire lifecycle of creating and managing LLM applications.

While Large Language Models (LLMs) are masters of language, their knowledge is static and they can’t interact with the outside world. They can’t check the current weather, perform a calculation, or look up information on a website. This is where the concept of Agents in LangChain comes in. An agent is a system that uses an LLM as its reasoning engine to decide which actions to take to accomplish a goal.

Machine learning can seem intimidating, often associated with complex mathematics and huge datasets. However, thanks to powerful libraries like Scikit-learn, training your first model is more accessible than ever. This guide will walk you through the fundamental steps of a machine learning project, from loading data to making predictions, using a classic example: classifying iris flowers.

1. Introduction to Hugging Face

Hugging Face is a platform that provides powerful tools and resources for the natural language processing (NLP) community. It offers various pre-trained models, APIs, and tools to simplify the development and deployment of NLP tasks for developers and researchers. The core mission of Hugging Face is to make NLP technology more accessible and shareable, promoting the advancement and democratization of artificial intelligence.

On Hugging Face, users can access a wide range of pre-trained models for tasks like text classification, question answering, language generation, and more. Additionally, Hugging Face allows users to create their own “Spaces” to store and share their models, code, and datasets. In this guide, we will focus on how to deploy a fast one-click face swapping software on Hugging Face.

2. Creating a Space and Uploading Code

To deploy a one-click face swapping software on Hugging Face, we first need to create a Hugging Face account and then create a new Space.

2.1 Create a Hugging Face Account

If you don’t have a Hugging Face account yet, head to the Hugging Face registration page and create a new account.

2.2 Login and Create a New Space

1. Login to Hugging Face, click the “Login” button in the top right corner, and enter your account credentials to log in.

2. After logging in, you will see your username in the top right corner. Click on your username and select “Create a new space.”. and select the Space SDK Gradio
   for space hardware, we will be using CPU.

3. On the “Create a new space” page, provide a name and description for your Space. Choose a relevant name related to your one-click face swapping software and provide an appropriate description to let others know about your project.

4. Click “Create new space” to finish creating the Space.
   

2.3 Upload Code and Applications

To deploy our one-click face swapping software, we will be using the roop repository available on GitHub. This software enables one-click face swapping for both images and videos by simply requiring users to upload a portrait image. For the purpose of this guide, we will focus on face swapping as an example. As we are using the free version of Hugging Face’s space, it currently supports CPU inference only.

To get started, clone the roop repository and create an app.py file.

we will call the core module in app.py using the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

# -* coding:UTF-8 -*
# !/usr/bin/env python
import numpy as np
import gradio as gr
import roop.globals
from roop.core import (
    start,
    decode_execution_providers,
    suggest_max_memory,
    suggest_execution_threads,
)
from roop.processors.frame.core import get_frame_processors_modules
from roop.utilities import normalize_output_path
import os
from PIL import Image


def swap_face(source_file, target_file):

    source_path = "input.jpg"
    target_path = "target.jpg"

    source_image = Image.fromarray(source_file)
    source_image.save(source_path)
    target_image = Image.fromarray(target_file)
    target_image.save(target_path)

    print("source_path: ", source_path)
    print("target_path: ", target_path)

    roop.globals.source_path = source_path
    roop.globals.target_path = target_path
    output_path = "output.jpg"
    roop.globals.output_path = normalize_output_path(
        roop.globals.source_path, roop.globals.target_path, output_path
    )
    roop.globals.frame_processors = ["face_swapper"]
    roop.globals.headless = True
    roop.globals.keep_fps = True
    roop.globals.keep_audio = True
    roop.globals.keep_frames = False
    roop.globals.many_faces = False
    roop.globals.video_encoder = "libx264"
    roop.globals.video_quality = 18
    roop.globals.max_memory = suggest_max_memory()
    roop.globals.execution_providers = decode_execution_providers(["cpu"])
    roop.globals.execution_threads = suggest_execution_threads()

    print(
        "start process",
        roop.globals.source_path,
        roop.globals.target_path,
        roop.globals.output_path,
    )

    for frame_processor in get_frame_processors_modules(
        roop.globals.frame_processors
    ):
        if not frame_processor.pre_check():
            return

    start()
    return output_path


app = gr.Interface(
    fn=swap_face, inputs=[gr.Image(), gr.Image()], outputs="image"
)
app.launch()

We will use Gradio to design the user interface for our program. The good news is that Hugging Face natively supports Gradio, so we don’t need to import any additional libraries.

Once you have written the program, you can push it to your Hugging Face space. After the push, the program will be automatically deployed. All you need to do is wait for the deployment to complete, and then you can start using the one-click face swapping software.

Remember, the deployment process is hassle-free and user-friendly, allowing you to focus on the exciting applications of your software.

If you’ve been exploring the world of Large Language Models (LLMs), you’ve likely heard of LangChain. It’s a powerful open-source framework designed to simplify the creation of applications powered by LLMs. LangChain provides a set of modular components and off-the-shelf “chains” that make it easy to build everything from simple prompt-response bots to complex, data-aware agents.

This guide will introduce you to the core concepts of LangChain and walk you through building your first LLM-powered application.

With the explosion of large language models (LLMs) and other deep learning applications, a new kind of data has become central to AI: vector embeddings. These dense numerical representations of text, images, or audio have one major problem—they are notoriously difficult to search efficiently. This is where a new type of database, the vector database, comes into play.