[ { "path": ".gitignore", "content": "# Created by https://www.toptal.com/developers/gitignore/api/pycharm\n# Edit at https://www.toptal.com/developers/gitignore?templates=pycharm\n\n### PyCharm ###\n# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider\n# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839\n\n# User-specific stuff\n.idea/**/workspace.xml\n.idea/**/tasks.xml\n.idea/**/usage.statistics.xml\n.idea/**/dictionaries\n.idea/**/shelf\n\n# AWS User-specific\n.idea/**/aws.xml\n\n# Generated files\n.idea/**/contentModel.xml\n\n# Sensitive or high-churn files\n.idea/**/dataSources/\n.idea/**/dataSources.ids\n.idea/**/dataSources.local.xml\n.idea/**/sqlDataSources.xml\n.idea/**/dynamic.xml\n.idea/**/uiDesigner.xml\n.idea/**/dbnavigator.xml\n\n# Gradle\n.idea/**/gradle.xml\n.idea/**/libraries\n\n# Gradle and Maven with auto-import\n# When using Gradle or Maven with auto-import, you should exclude module files,\n# since they will be recreated, and may cause churn. Uncomment if using\n# auto-import.\n# .idea/artifacts\n# .idea/compiler.xml\n# .idea/jarRepositories.xml\n# .idea/modules.xml\n# .idea/*.iml\n# .idea/modules\n# *.iml\n# *.ipr\n\n# CMake\ncmake-build-*/\n\n# Mongo Explorer plugin\n.idea/**/mongoSettings.xml\n\n# File-based project format\n*.iws\n\n# IntelliJ\nout/\n\n# mpeltonen/sbt-idea plugin\n.idea_modules/\n\n# JIRA plugin\natlassian-ide-plugin.xml\n\n# Cursive Clojure plugin\n.idea/replstate.xml\n\n# SonarLint plugin\n.idea/sonarlint/\n\n# Crashlytics plugin (for Android Studio and IntelliJ)\ncom_crashlytics_export_strings.xml\ncrashlytics.properties\ncrashlytics-build.properties\nfabric.properties\n\n# Editor-based Rest Client\n.idea/httpRequests\n\n# Android studio 3.1+ serialized cache file\n.idea/caches/build_file_checksums.ser\n\n### PyCharm Patch ###\n# Comment Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-215987721\n\n# *.iml\n# modules.xml\n# .idea/misc.xml\n# *.ipr\n\n# Sonarlint plugin\n# https://plugins.jetbrains.com/plugin/7973-sonarlint\n.idea/**/sonarlint/\n\n# SonarQube Plugin\n# https://plugins.jetbrains.com/plugin/7238-sonarqube-community-plugin\n.idea/**/sonarIssues.xml\n\n# Markdown Navigator plugin\n# https://plugins.jetbrains.com/plugin/7896-markdown-navigator-enhanced\n.idea/**/markdown-navigator.xml\n.idea/**/markdown-navigator-enh.xml\n.idea/**/markdown-navigator/\n\n# Cache file creation bug\n# See https://youtrack.jetbrains.com/issue/JBR-2257\n.idea/$CACHE_FILE$\n\n# CodeStream plugin\n# https://plugins.jetbrains.com/plugin/12206-codestream\n.idea/codestream.xml\n\n# Azure Toolkit for IntelliJ plugin\n# https://plugins.jetbrains.com/plugin/8053-azure-toolkit-for-intellij\n.idea/**/azureSettings.xml\n\n# End of https://www.toptal.com/developers/gitignore/api/pycharm\nAsk AI to edit or generate...\n" }, { "path": ".idea/.gitignore", "content": "# Default ignored files\n/shelf/\n/workspace.xml\n# Editor-based HTTP Client requests\n/httpRequests/\n# Datasource local storage ignored files\n/dataSources/\n/dataSources.local.xml\n" }, { "path": ".idea/inspectionProfiles/profiles_settings.xml", "content": "\n \n \n" }, { "path": ".idea/misc.xml", "content": "\n\n \n" }, { "path": ".idea/modules.xml", "content": "\n\n \n \n \n \n \n" }, { "path": ".idea/python advanced topics.iml", "content": "\n\n \n \n \n \n \n" }, { "path": ".idea/vcs.xml", "content": "\n\n \n \n \n" }, { "path": "README.md", "content": "![](image1.png)\n\n## Introduction\n\nIn this research, I have given a brief overview of advanced topics in Python and Django. This article does not provide\nin-depth knowledge on various topics and the purpose of this research is to collect topics in one place to provide the\nreader with a mental framework. In this article, I have used very simple examples to explain the topics easily.\nNaturally, studying and examining more and more practical examples will help a lot to understand each topic. I hope\nreading this article is useful for you. (**Hamid Asgari**)\n\n\n\n\n * [Introduction](#introduction)\n * [Python related topics:](#python-related-topics)\n * [object-oriented programming (OOP)](#object-oriented-programming-oop)\n * [Inheritance](#inheritance)\n * [Method Resolution Order (MRO)](#method-resolution-order-mro)\n * [1. Single Inheritance](#1-single-inheritance)\n * [2. Multiple Inheritance](#2-multiple-inheritance)\n * [3. The Diamond Inheritance Problem](#3-the-diamond-inheritance-problem)\n * [4. Using super() with MRO](#4-using-super-with-mro)\n * [5. Checking the MRO](#5-checking-the-mro)\n * [Polymorphism](#polymorphism)\n * [Encapsulation](#encapsulation)\n * [Abstraction](#abstraction)\n * [Decorators](#decorators)\n * [More decorators in Python](#more-decorators-in-python)\n * [Map function](#map-function)\n * [Itertools](#itertools)\n * [Lambda function](#lambda-function)\n * [Exception Handling](#exception-handling)\n * [SOLID principles](#solid-principles)\n * [Python collection](#python-collection)\n * [frozenset in Python](#frozenset-in-python)\n * [Generators and Iterators](#generators-and-iterators)\n * [Magic methods](#magic-methods)\n * [GIL](#gil)\n * [concurrency and parallelism](#concurrency-and-parallelism)\n * [Main types of methods in python classes](#main-types-of-methods-in-python-classes)\n * [Data serialization](#data-serialization)\n * [Data class in python](#data-class-in-python)\n * [Shallow copy and deep copy](#shallow-copy-and-deep-copy)\n * [Local and global variables](#local-and-global-variables)\n * [Comprehension](#comprehension)\n * [Pydantic](#pydantic)\n * [Args and Kwargs in Python](#args-and-kwargs-in-python)\n * [Operator overloading](#operator-overloading)\n * [Recursive function](#recursive-function)\n * [Context manager](#context-manager)\n * [Python 3.11 over previous versions](#python-311-over-previous-versions)\n * [Semaphores and Mutexes](#semaphores-and-mutexes)\n * [Python built-in functions](#python-built-in-functions)\n * [Type Hints and Type Checking](#type-hints-and-type-checking)\n * [Package Management](#package-management)\n * [functools Module](#functools-module)\n * [Advanced Collections](#advanced-collections)\n * [Enum and NamedTuple](#enum-and-namedtuple)\n * [asyncio (Advanced)](#asyncio-advanced)\n * [datetime Module](#datetime-module)\n * [Abstract Base Classes (ABC) Advanced](#abstract-base-classes-abc-advanced)\n * [JSON vs Pickle Comparison](#json-vs-pickle-comparison)\n * [Composition vs Inheritance](#composition-vs-inheritance)\n * [Django related topics:](#django-related-topics)\n * [Django signals](#django-signals)\n * [Django middleware](#django-middleware)\n * [Django custom template tags](#django-custom-template-tags)\n * [Django permissions](#django-permissions)\n * [Django custom user models](#django-custom-user-models)\n * [Django Custom Managers](#django-custom-managers)\n * [Django Custom validators](#django-custom-validators)\n * [Custom management commands](#custom-management-commands)\n * [Django's Query API](#djangos-query-api)\n * [Custom query expressions or Custom Lookup](#custom-query-expressions-or-custom-lookup)\n * [Django Filterset](#django-filterset)\n * [Context managers](#context-managers)\n * [Django Channels](#django-channels)\n * [HTTP methods in Django](#http-methods-in-django)\n * [annotate and aggregate in Django](#annotate-and-aggregate-in-django)\n * [Mixin in Django](#mixin-in-django)\n * [Cache in Django](#cache-in-django)\n * [Django constraint](#django-constraint)\n * [bulk creation in Django](#bulk-creation-in-django)\n * [prefetch_related and select_related in Django](#prefetch_related-and-select_related-in-django)\n * [Third-party packages in Django](#third-party-packages-in-django)\n * [Property decorators](#property-decorators)\n * [WSGI and ASGI](#wsgi-and-asgi)\n * [WSGI (Web Server Gateway Interface):](#wsgi-web-server-gateway-interface)\n * [ASGI (Asynchronous Server Gateway Interface):](#asgi-asynchronous-server-gateway-interface)\n * [Advanced features in ORM](#advanced-features-in-orm)\n * [Class-based views methods](#class-based-views-methods)\n * [Django optimization](#django-optimization)\n * [Generic Foreign Key in Django](#generic-foreign-key-in-django)\n * [Django custom exceptions](#django-custom-exceptions)\n * [select_for_update in Django](#select_for_update-in-django)\n * [Django model methods](#django-model-methods)\n * [Parametric unit tests](#parametric-unit-tests)\n * [Testing in Django](#testing-in-django)\n * [Security Best Practices in Django](#security-best-practices-in-django)\n * [Logging and Monitoring in Django](#logging-and-monitoring-in-django)\n * [FastAPI related topics:](#fastapi-related-topics)\n * [Dependency Injection in FastAPI](#dependency-injection-in-fastapi)\n * [use cases](#use-cases)\n * [Dependency Ordering](#dependency-ordering)\n * [Pydantic methods in FastAPI](#pydantic-methods-in-fastapi)\n * [Decorators in FastAPI](#decorators-in-fastapi)\n * [Real-World Use Cases of decorators:](#real-world-use-cases-of-decorators)\n * [WebSockets in FastAPI](#websockets-in-fastapi)\n * [Real-World Use Cases of websocket:](#real-world-use-cases-of-websocket)\n * [Asynchronous File Uploads](#asynchronous-file-uploads)\n * [Real-World Use Cases of Asynchronous File Uploads:](#real-world-use-cases-of-asynchronous-file-uploads)\n * [Security Headers](#security-headers)\n * [Background Tasks](#background-tasks)\n * [Middleware in FastAPI](#middleware-in-fastapi)\n * [Real-World Use Cases of Middlewares:](#real-world-use-cases-of-middlewares)\n * [Permissions in FastAPI](#permissions-in-fastapi)\n * [Custom Validators in FastAPI](#custom-validators-in-fastapi)\n * [Use Cases in Real-World Applications](#use-cases-in-real-world-applications)\n * [FastAPI BaseSettings](#fastapi-basesettings)\n * [The use of **BaseSettings** in FastAPI has several real-world applications, including:](#the-use-of-basesettings-in-fastapi-has-several-real-world-applications-including)\n * [Dependency Caching](#dependency-caching)\n * [Use Cases in Real-World Applications](#use-cases-in-real-world-applications-1)\n * [Rate Limiting](#rate-limiting)\n * [Use Cases in Real-World Applications](#use-cases-in-real-world-applications-2)\n * [Cache in FastAPI](#cache-in-fastapi)\n * [Use Cases in Real-World Applications](#use-cases-in-real-world-applications-3)\n * [Custom Exceptions in FastAPI](#custom-exceptions-in-fastapi)\n * [Use Cases in Real-World Applications](#use-cases-in-real-world-applications-4)\n * [Optimization techniques in FastAPI](#optimization-techniques-in-fastapi)\n * [Concurrency and Parallelism In FastAPI](#concurrency-and-parallelism-in-fastapi)\n * [API Documentation Best Practices](#api-documentation-best-practices)\n * [Testing in FastAPI](#testing-in-fastapi)\n * [Security Best Practices in FastAPI](#security-best-practices-in-fastapi)\n * [Logging and Monitoring in FastAPI](#logging-and-monitoring-in-fastapi)\n * [General Topics:](#general-topics)\n * [REST API Design Principles](#rest-api-design-principles)\n * [Deployment and DevOps](#deployment-and-devops)\n * [Docker and Containerization](#docker-and-containerization)\n * [CI/CD Pipelines](#cicd-pipelines)\n * [Environment Management](#environment-management)\n * [Interview Preparation questions](#interview-preparation-questions)\n * [PostgreSQL Querying](#postgresql-querying)\n * [Algorithmic Problem Solving](#algorithmic-problem-solving)\n * [Common Data Structures Review](#common-data-structures-review)\n * [Problem-Solving Strategy](#problem-solving-strategy)\n * [Practice Problems & Solutions (Python)](#practice-problems--solutions-python)\n * [Interview Questions](#interview-questions)\n * [Problem 1: Database Transactions (Django & SQLAlchemy)](#problem-1-database-transactions-django--sqlalchemy)\n\n\n## Python related topics:\n\n### object-oriented programming (OOP)\n\nHere are some of the most important topics in object-oriented programming (**OOP**) in Python, along with simple\nexamples of each:\n\n#### Inheritance\n\nInheritance is a mechanism that allows a class to inherit properties and methods from another class. The class that\ninherits from another class is called a subclass or derived class, while the class that is being inherited from is\ncalled a superclass or base class.\n\n```\nfrom abc import ABC, abstractmethod\n\nclass Animal(ABC):\n def __init__(self, name):\n self.name = name\n\n @abstractmethod\n def speak(self):\n pass\n\nclass Dog(Animal):\n def speak(self):\n return \"Woof!\"\n\nclass Cat(Animal):\n def speak(self):\n return \"Meow!\"\n\ndog = Dog(\"Fido\")\nprint(dog.name) # Output: Fido\nprint(dog.speak()) # Output: Woof!\n\ncat = Cat(\"Whiskers\")\nprint(cat.name) # Output: Whiskers\nprint(cat.speak()) # Output: Meow!\n\n```\n\nIn this example, Animal is the superclass, and Dog and Cat are subclasses that inherit from it. The speak method is an\nabstract method in the Animal class that is implemented in the subclasses.\nWe import the **ABC** (Abstract Base Class) class from the **abc** module and use the **@abstractmethod** decorator to mark the **speak** method as an abstract method in the **Animal** class. This enforces that subclasses of **Animal** must provide an implementation for the speak method.\n\n#### Method Resolution Order (MRO)\n\nThe method resolution order (MRO) defines the sequence in which base classes are searched when executing a method. Initially, the search starts within the class itself, and then proceeds according to the order specified during inheritance. This order is also known as the linearization of a class, and the set of rules governing it is referred to as MRO (Method Resolution Order).\n\nTo illustrate this concept more clearly, let's walk through some examples that cover different inheritance scenarios.\n\n##### 1. Single Inheritance\n\nIn single inheritance, the MRO is very straightforward. When a derived class inherits from a single base class, Python searches the base class for the desired method or attribute if it’s not found in the derived class.\n\n```\nclass A:\n def greet(self):\n print(\"Hello from A\")\n\nclass B(A):\n pass\n\nb = B()\nb.greet()\n```\n\nOutput:\n\n```\nHello from A\n```\n\n##### 2. Multiple Inheritance\n\nIn multiple inheritance, the MRO determines the sequence in which parent classes are called. Python uses the C3 linearization algorithm to establish this order, ensuring that each class appears only once.\n\n```\nclass A:\n def greet(self):\n print(\"Hello from A\")\n\nclass B:\n def greet(self):\n print(\"Hello from B\")\n\nclass C(A, B):\n pass\n\nc = C()\nc.greet()\n```\n\nOutput:\n\n```\nHello from A\n```\n\n##### 3. The Diamond Inheritance Problem\n\nThe diamond inheritance problem occurs when a class inherits from multiple classes that share a common ancestor. MRO ensures that the common ancestor is not invoked multiple times, preventing redundancy.\n\n```\nclass A:\n def greet(self):\n print(\"Hello from A\")\n\nclass B(A):\n def greet(self):\n print(\"Hello from B\")\n\nclass C(A):\n def greet(self):\n print(\"Hello from C\")\n\nclass D(B, C):\n pass\n\nd = D()\nd.greet()\n```\n\nOutput:\n\n```\nHello from B\n```\n\n##### 4. Using super() with MRO\n\nUsing the super() function in a multiple inheritance scenario ensures that the next class in the MRO sequence is called. This is particularly useful for maintaining a consistent flow through all classes in the hierarchy.\n\n```\nclass A:\n def greet(self):\n print(\"Hello from A\")\n\nclass B(A):\n def greet(self):\n print(\"Hello from B\")\n super().greet()\n\nclass C(A):\n def greet(self):\n print(\"Hello from C\")\n super().greet()\n\nclass D(B, C):\n def greet(self):\n print(\"Hello from D\")\n super().greet()\n\nd = D()\nd.greet()\n```\n\nExplanation:\n\nD calls B, B calls C, and C calls A according to the MRO (D -> B -> C -> A).\n\nUsing super() ensures that each class in the MRO chain is called in order.\n\nOutput:\n\n```\nHello from D\nHello from B\nHello from C\nHello from A\n```\n\n##### 5. Checking the MRO\n\nYou can inspect the MRO of a class using the mro() method or the __mro__ attribute.\n\n```\nprint(D.mro())\n```\n\nOutput:\n\n```\n[, , , , ]\n```\n\nThis output tells you the exact order in which classes are searched for methods or attributes when an instance of D is used.\n\n#### Polymorphism\n\nPolymorphism is the ability of objects to take on different forms or perform different actions depending on the context.\nIn Python, polymorphism is achieved through **method overriding** and **method overloading**.\n\n```\nfrom abc import ABC, abstractmethod\nimport math\n\nclass Shape(ABC):\n @abstractmethod\n def area(self):\n pass\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n\n def area(self):\n return self.width * self.height\n\nclass Circle(Shape):\n def __init__(self, radius):\n self.radius = radius\n\n def area(self):\n return math.pi * self.radius ** 2\n\nshapes = [Rectangle(5, 10), Circle(7)]\nfor shape in shapes:\n print(shape.area())\n\n```\n\nIn this example, Shape is an abstract class that defines the area method as an abstract method. Rectangle and Circle are\nconcrete subclasses that implement the area method. The shapes list contains instances of both Rectangle and Circle, and\nthe area method is called on each instance, demonstrating polymorphism.\n\nIn this code, we use the **ABC** class and the **@abstractmethod** decorator to define **Shape** as an abstract base class with an abstract area method. This ensures that subclasses of **Shape** must implement the area method. The **math.pi** constant is used for more accurate calculations of the circle's area.\n\n#### Encapsulation\n\nEncapsulation is the practice of hiding the internal details of an object and providing a public interface for\ninteracting with it. In Python, encapsulation is achieved through the use of _public and private methods and attributes_.\n\n```\nclass BankAccount:\n def __init__(self, balance):\n self._balance = balance\n\n def deposit(self, amount):\n self._balance += amount\n\n def withdraw(self, amount):\n if self._balance >= amount:\n self._balance -= amount\n else:\n raise ValueError(\"Insufficient balance\")\n\n def get_balance(self):\n return self._balance\n\n\naccount = BankAccount(1000)\naccount.deposit(500)\naccount.withdraw(200)\nprint(account.get_balance()) # Output: 1300\n```\n\nIn this example, **BankAccount** is a class that represents a bank account. The **_balance** attribute is marked as _private_\nby convention (_using a single underscore_), and can only be accessed through public methods such as **deposit**, **withdraw**,\nand **get_balance**.\n\n#### Abstraction\n\nAbstraction is the process of simplifying complex systems by breaking them down into smaller, more manageable parts. In\nPython, abstraction is achieved through the use of _abstract classes_ and interfaces.\n\n```\nfrom abc import ABC, abstractmethod\n\nclass Shape(ABC):\n @abstractmethod\n def area(self):\n pass\n\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n\n def area(self):\n return self.width * self.height\n\n\nshapes = [Rectangle(5, 10)]\nfor shape in shapes:\n print(shape.area())\n```\n\nIn this example, **Shape** is an abstract class that defines the **area** method as an **abstract** method. **Rectangle** is a concrete\nsubclass that implements the area method. The shapes list contains an instance of **Rectangle**, demonstrating abstraction.\n\n### Decorators\n\nIn Python, decorators are a way to modify the behavior of functions or classes by wrapping them with other functions.\nHere's a simple example to demonstrate the basic concept of decorators:\n\n```\ndef decorator_function(original_function):\n def wrapper_function():\n print(\"Before the original function is called.\")\n original_function()\n print(\"After the original function is called.\")\n return wrapper_function\n\n@decorator_function\ndef greet():\n print(\"Hello, world!\")\n\ngreet()\n```\n\nout put\n\n```\nBefore the original function is called.\nHello, world!\nAfter the original function is called.\n```\n\nHere is an example of how to implement the previous decorator in a class-based way:\n\n```angular2html\nclass DecoratorClass:\n def __init__(self, original_function):\n self.original_function = original_function\n\n def __call__(self):\n print(\"Before the original function is called.\")\n self.original_function()\n print(\"After the original function is called.\")\n\n@DecoratorClass\ndef greet():\n print(\"Hello, world!\")\n\ngreet()\n```\n\nIn this example, the **DecoratorClass** is a class-based decorator that takes the original function as an argument in its constructor. The **__call__** method is used to define the behavior of the decorator when the decorated function is called.\n\n### More decorators in Python\n\nIn Python, class decorators are functions or callable objects that are used to modify or enhance the behavior of a class. They are applied to the class definition itself and can add or modify class-level attributes, methods, or perform other operations on the class.\nHere are a few examples of class decorators:\n\n- classmethod\n\nThe **classmethod** decorator is a built-in decorator that transforms a method into a class method. Class methods can be called on the class itself, rather than on instances of the class.\nHere's a simple example of a class method in Python\n\n```\nclass Car:\n total_cars = 0 # Class variable to keep track of the total number of cars\n\n def __init__(self, name):\n self.name = name\n Car.total_cars += 1\n\n def get_name(self):\n return self.name\n\n @classmethod\n def get_total_cars(cls):\n return cls.total_cars\n\n\ncar1 = Car(\"Toyota\")\ncar2 = Car(\"Honda\")\ncar3 = Car(\"Ford\")\n\nprint(car1.get_name()) # Output: \"Toyota\"\nprint(Car.get_total_cars()) # Output: 3\n```\n\nIn this example, we have a **Car** class with a class variable **total_cars**, which keeps track of the total number of car objects created. The **__init__** method increments **total_cars** by 1 whenever a new car object is created.\n\n- staticmethod\n\nThe **staticmethod** decorator is another built-in decorator that transforms a method into a static method. Static methods are similar to regular functions and do not have access to the class or instance.\nHere's a simple example of a static method in Python:\n\n```angular2html\nclass MathUtils:\n @staticmethod\n def add_numbers(x, y):\n return x + y\n\n @staticmethod\n def multiply_numbers(x, y):\n return x * y\n\n\nsum_result = MathUtils.add_numbers(5, 3)\nprint(sum_result) # Output: 8\n\nproduct_result = MathUtils.multiply_numbers(4, 2)\nprint(product_result) # Output: 8\n```\n\nStatic methods are defined using the **@staticmethod** decorator. They don't receive any special first parameter like instance methods or class methods. They behave like regular functions defined within the class.\n\n- property\n\nThe **property** decorator allows you to define methods that can be accessed like attributes, providing a way to implement computed or dynamic properties.\n\n```angular2html\nclass Circle:\n def __init__(self, radius):\n self.radius = radius\n\n @property\n def diameter(self):\n return self.radius * 2\n\n @property\n def area(self):\n return 3.14 * self.radius**2\n\n# Creating an instance of Circle\nmy_circle = Circle(5)\n\nprint(my_circle.diameter) # Output: 10\nprint(my_circle.area) # Output: 78.5\n```\n\nIn this example, we use the **property** decorator to define the **diameter** and **area** methods.\n\n- abstractmethod\n\nThe **abstractmethod** decorator is used in combination with the **ABC** module to define abstract methods in abstract base classes\nThe **abstractmethod** decorator ensures that any subclass of super class must implement this method.\n\n```\nfrom abc import ABC, abstractmethod\n\nclass Shape(ABC):\n @abstractmethod\n def calculate_area(self):\n pass\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n\n def calculate_area(self):\n return self.width * self.height\n\n# Creating an instance of Rectangle\nmy_rectangle = Rectangle(5, 10)\n\nprint(my_rectangle.calculate_area()) # Output: 50\n```\n\nThe **abstractmethod** decorator ensures that any subclass of **Shape** must implement this method.\n\n- dataclass\n\nThe **dataclass** decorator is available in the **dataclasses** module (introduced in Python 3.7) and provides a concise way to define classes that are primarily used to store data.\n\n```angular2html\nfrom dataclasses import dataclass\n\n@dataclass\nclass Person:\n name: str\n age: int\n\n# Creating an instance of Person\nperson = Person(\"Alice\", 25)\n\nprint(person.name) # Output: Alice\nprint(person.age) # Output: 25\n```\n\nIn this example, we apply the **@dataclass** decorator to the **Person** class. The decorator automatically generates special methods like **__init__, __repr__, and __eq__** based on the class variables (name and age in this case).\nThe **dataclass** decorator simplifies the process of defining classes that are primarily used for holding data, reducing the amount of boilerplate code required.\n\n- cached_property\n\nThe **cached_property** decorator is available in third-party libraries such as **django.utils.functional** and **cachetools** and provides a way to cache the result of a method as a property, improving performance when the method is called multiple times.\n\n```angular2html\nfrom django.utils.functional import cached_property\n\nclass Square:\n def __init__(self, side):\n self.side = side\n\n @cached_property\n def area(self):\n print(\"Calculating area...\")\n return self.side**2\n\n# Creating an instance of Square\nmy_square = Square(5)\n\nprint(my_square.area) # Output: Calculating area... 25\nprint(my_square.area) # Output: 25 (Cached result, no recalculation)\n```\n\nIn this example, we apply the **@cached_property** decorator to the **area** method. The decorator caches the result of the method on the instance, so subsequent accesses to area return the cached value without recalculating it.\n\n### Map function\n\nThe __map()__ function in Python is a built-in function that applies a given function to each item in an iterable (such as a\nlist, tuple, or string) and returns an iterator with the results. It takes two or more arguments: the __function__ to apply\nand one or more iterables. The basic syntax of the __map()__ function is as follows:\n\n```\nmap(function, iterable1, iterable2, ...)\n```\n\nHere's an example that demonstrates the usage of the **map()** function:\n\n```\n# Example 1: Squaring numbers using map()\nnumbers = [1, 2, 3, 4, 5]\nsquared = map(lambda x: x**2, numbers)\nprint(list(squared))\n# Output: [1, 4, 9, 16, 25]\n```\n\n### Itertools\n\nItertools is a Python module that provides a collection of functions for creating and manipulating iterators, which are\nobjects that can be iterated (looped) over. Here are some commonly used functions provided by the itertools module\n\n- **count()**: Generates an infinite sequence of numbers starting from a specified value.\n- **chain()**: Combines multiple iterators into a single iterator.\n- **groupby()**: Groups consecutive elements in an iterable based on a common key\n- **cycle()**: cycles through the elements of an iterable object infinitely\n- **combinations()**: generates all possible combinations of a given iterable object\n\nexample of cycle():\n\n```\nimport itertools\n\ncolors = ['red', 'green', 'blue']\ncolor_cycle = itertools.cycle(colors)\n\nfor _ in range(5):\n print(next(color_cycle))\n```\n\nout put:\n\n```\nred\ngreen\nblue\nred\ngreen\n```\n\nexample of combinations():\n\n```\nimport itertools\n\nnumbers = [1, 2, 3]\ncombinations = itertools.combinations(numbers, 2)\n\nfor combination in combinations:\n print(combination)\n```\n\noutput:\n\n```\n(1, 2)\n(1, 3)\n(2, 3)\n```\n\nHere are a few examples of how you can utilize itertools in the context of Django. In the following example, we use\nitertools.chain() to combine the querysets of posts and comments into a single iterable. We then sort the combined\nresults based on the creation date, using sorted().\n\n```\nimport itertools\nfrom myapp.models import Post, Comment\n\nposts = Post.objects.filter(published=True)\ncomments = Comment.objects.filter(approved=True)\n\ncombined_results = itertools.chain(posts, comments)\nsorted_results = sorted(combined_results, key=lambda item: item.created_at, reverse=True)\n```\n\n### Lambda function\n\nIn Python, a **lambda** function is a small anonymous function that can be defined without a name. It is also known as an\ninline function or a lambda expression. Lambda functions are typically used when you need a simple function that will be\nused only once or as a parameter to another function. The general syntax of a lambda function in Python is:\n\n```\nlambda arguments: expression\n```\n\nFor example, let's say we want to create a lambda function that takes two arguments and returns their sum:\n\n```\nadd = lambda x, y: x + y\nresult = add(3, 5)\nprint(result) # Output: 8\n```\n\nLambda functions are often used with built-in functions like **map()**, **filter()**, and **reduce()**. These functions take another\nfunction as a parameter, and lambda functions provide a convenient way to define these functions on the fly without\nexplicitly defining a separate function. Here's an example using the **map()** function with a **lambda** function to square a\nlist of numbers:\n\n```\nnumbers = [1, 2, 3, 4, 5]\nsquared_numbers = list(map(lambda x: x ** 2, numbers))\nprint(squared_numbers) # Output: [1, 4, 9, 16, 25]\n```\n\nFor more complex or reusable functions, it's often better to define a regular named function using the **def** keyword.\n\n### Exception Handling\n\nException handling in Python allows you to gracefully handle and recover from runtime errors or exceptional conditions\nthat may occur during the execution of your program. Here's an example that demonstrates the usage of exception\nhandling:\n\n```\ntry:\n # Code that may raise an exception\n x = 10 / 0 # division by zero raises a ZeroDivisionError\nexcept ZeroDivisionError:\n # Code to handle ZeroDivisionError\n print(\"Division by zero is not allowed.\")\n\n\n# Output:\n# Division by zero is not allowed.\n```\n\n### SOLID principles\n\nThe SOLID principles are a set of design principles that help in creating maintainable, scalable, and flexible software\nsystems.\n\n- **Single Responsibility Principle (SRP):**\n\nThis principle states that a class should have only one responsibility. Let's say we have a **User** class that handles both user authentication and user data management.\nInstead, we can separate these responsibilities into two classes: **Authenticator** and **UserManager**. Here's a Python\nexample:\n\n```\nclass Authenticator:\n def authenticate(self, username, password):\n # Authentication logic here\n\nclass UserManager:\n def create_user(self, user_data):\n # User creation logic here\n\n def delete_user(self, user_id):\n # User deletion logic here\n```\n\n- **Open/Closed Principle (OCP):**\n\n This principle states that software entities (classes, modules, functions) should be open for extension but closed for\n modification. In other words, you should be able to add new functionality without modifying existing code. Let's say\n we have a **Shape** class with different subclasses such as **Circle** and **Rectangle**. Instead of modifying the **Shape** class\n every time we want to add a new shape, we can use **inheritance** and **polymorphism** to extend the functionality:\n\n```\nclass Shape:\n def area(self):\n raise NotImplementedError()\n\nclass Circle(Shape):\n def __init__(self, radius):\n self.radius = radius\n\n def area(self):\n return 3.14 * self.radius * self.radius\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n\n def area(self):\n return self.width * self.height\n```\n\n- **Liskov Substitution Principle (LSP):**\n\n This principle states that objects of a superclass should be replaceable with objects of its subclasses without\n affecting the correctness of the program. Let's say we have a function that expects a **Shape** object. According to LSP,\n we should be able to pass any subclass of **Shape** without causing any issues. In simpler terms, it means that a subclass should be able to be used wherever its superclass is expected, without causing any issues or breaking the functionality of the program.\n\nHere's a simple Python example to illustrate the _Liskov Substitution Principle_:\n\n```angular2html\nclass Shape:\n def area(self):\n pass\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n\n def area(self):\n return self.width * self.height\n\nclass Square(Shape):\n def __init__(self, side):\n self.side = side\n\n def area(self):\n return self.side * self.side\n```\n\nIn this example, we have a superclass called **Shape** with a method **area()** that calculates the area of a shape. We then have two subclasses, **Rectangle** and **Square**, that inherit from the **Shape** class and override the **area()** method to calculate the area specific to each shape.\nAccording to the _Liskov Substitution Principle_, we should be able to use objects of the **Rectangle** and **Square** classes interchangeably with objects of the Shape class. For example:\n\n```angular2html\ndef print_area(shape):\n print(f\"The area is: {shape.area()}\")\n\nrectangle = Rectangle(4, 5)\nsquare = Square(4)\n\nprint_area(rectangle) # Output: The area is: 20\nprint_area(square) # Output: The area is: 16\n```\n\nIn this example, we can see that both the **Rectangle** and **Square** objects can be passed to the **print_area()** function, which expects an object of the **Shape** class. This demonstrates the _Liskov Substitution Principle_, as the subclasses can be used in place of the superclass without any issues or breaking the functionality of the program.\n\n- **Interface Segregation Principle (ISP):**\n\nThis principle states that clients should not be forced to depend on interfaces they do not use. It promotes the idea of\nhaving smaller, focused interfaces rather than large, general-purpose ones. Let's say we have an **Animal** interface with\nmethods like **walk(), swim(), and fly()**. Instead of having a single interface **Animal**, we can split it into smaller interfaces\nbased on functionality:\n\n```\nclass Walker:\n def walk(self):\n raise NotImplementedError()\n\nclass Swimmer:\n def swim(self):\n raise NotImplementedError()\n\nclass Flyer:\n def fly(self):\n raise NotImplementedError()\n\nclass Dog(Walker):\n def walk(self):\n print(\"Dog is walking\")\n\nclass Fish(Swimmer):\n def swim(self):\n print(\"Fish is swimming\")\n\n```\n\n- **Dependency Inversion Principle (DIP):**\n\n This principle states that high-level modules should not depend on low-level modules. Both should depend on\n abstractions. It encourages the use of interfaces or abstract classes to decouple dependencies. . This helps to decouple the high-level and low-level modules, making it easier to change the low-level ones without affecting the high-level ones\n\n### Python collection\n\nPython provides several built-in collection types, such as **lists, tuples, sets, and dictionaries**. These collections can\nbe used to store and organize data efficiently.\n\n- **lists**:\n\n A list is a mutable collection that can store an ordered sequence of elements. It is defined using square\n brackets (**[]**). Here's an example:\n\n```\n# Creating a list\nfruits = ['apple', 'banana', 'orange']\n\n# Accessing elements\nprint(fruits[0]) # Output: apple\n\n# Modifying elements\nfruits[1] = 'kiwi'\nprint(fruits) # Output: ['apple', 'kiwi', 'orange']\n\n# Adding elements\nfruits.append('grape')\nprint(fruits) # Output: ['apple', 'kiwi', 'orange', 'grape']\n\n# Removing elements\nfruits.remove('kiwi')\nprint(fruits) # Output: ['apple', 'orange', 'grape']\n\n```\n\n- **Tuples**:\n\n A tuple is an immutable collection that can store an ordered sequence of elements. It is defined using\n parentheses (). Here's an example:\n\n```\n# Creating a tuple\npoint = (3, 4)\n\n# Accessing elements\nprint(point[0]) # Output: 3\n\n# Unpacking tuple\nx, y = point\nprint(x, y) # Output: 3 4\n\n```\n\n- **Sets**:\n\n A set is an unordered collection that stores unique elements. It is defined using curly braces ({}) or the set()\n function. Here's an example:\n\n```\n# Creating a set\nnumbers = {1, 2, 3, 4}\n\n# Adding elements\nnumbers.add(5)\nprint(numbers) # Output: {1, 2, 3, 4, 5}\n\n# Removing elements\nnumbers.remove(2)\nprint(numbers) # Output: {1, 3, 4, 5}\n\n```\n\n- **Dictionaries**:\n\n A dictionary is a collection that stores **key-value** pairs. It is defined using curly braces ({}) and colons (:). Here's\n an example:\n\n```\n# Creating a dictionary\nstudent = {\n 'name': 'Alice',\n 'age': 20,\n 'university': 'XYZ'\n}\n\n# Accessing values\nprint(student['name']) # Output: Alice\n\n# Modifying values\nstudent['age'] = 21\nprint(student) # Output: {'name': 'Alice', 'age': 21, 'university': 'XYZ'}\n\n# Adding new key-value pair\nstudent['major'] = 'Computer Science'\nprint(student) # Output: {'name': 'Alice', 'age': 21, 'university': 'XYZ', 'major': 'Computer Science'}\n\n# Removing key-value pair\ndel student['university']\nprint(student) # Output: {'name': 'Alice', 'age': 21, 'major': 'Computer Science'}\n\n# Updating a field\nstudent.update({'age': 21})\n\n```\n\n### frozenset in Python\n\nIn Python, a **frozenset** is an immutable (unchangeable) version of the built-in set type. It is an unordered collection of unique elements, just like a regular set, but it cannot be modified once created. This means you cannot add, remove, or modify elements in a frozenset after it is created.\nHere's an example of creating and utilizing a frozenset:\n\n```angular2html\n# Create a frozenset\nnumbers = frozenset([1, 2, 3, 4, 5])\n\n# Accessing elements\nfor number in numbers:\n print(number)\n\n# Frozenset operations\nother_numbers = frozenset([4, 5, 6, 7, 8])\n\n# Union\nunion = numbers.union(other_numbers)\nprint(union) # Output: frozenset({1, 2, 3, 4, 5, 6, 7, 8})\n\n# Intersection\nintersection = numbers.intersection(other_numbers)\nprint(intersection) # Output: frozenset({4, 5})\n\n# Difference\ndifference = numbers.difference(other_numbers)\nprint(difference) # Output: frozenset({1, 2, 3})\n\n# Subset check\nis_subset = numbers.issubset(other_numbers)\nprint(is_subset) # Output: False\n```\n\n### Generators and Iterators\n\nGenerators and iterators are powerful constructs in Python used for efficient iteration and lazy evaluation. They\nprovide a way to generate or yield values on the fly, enabling you to work with large or infinite sequences without\nloading everything into memory at once.\n\n- Lazy file reading using generators:\n\n```\ndef read_lines(filename):\n with open(filename, 'r') as file:\n for line in file:\n yield line.strip()\n\nlines = read_lines('large_file.txt')\nfor line in lines:\n print(line)\n\n```\n\nHere, the **read_lines()** function returns a generator that yields one line at a time. This approach is memory-efficient\nand allows you to process large files without loading the entire contents into memory.\nHere is the implementation of the previous example with custom iterator:\n\n```\nclass LineIterator:\n def __init__(self, filename):\n self.filename = filename\n self.file = None\n\n def __iter__(self):\n self.file = open(self.filename, 'r')\n return self\n\n def __next__(self):\n line = self.file.readline().strip()\n if not line:\n self.file.close()\n raise StopIteration\n return line\n\nlines = LineIterator('large_file.txt')\nfor line in lines:\n print(line)\n\n```\n\nThe main advantage of the generator approach is its simplicity and conciseness. The generator function hides most of the\nimplementation details required for iterators, resulting in cleaner code. On the other hand, the iterator approach\nprovides more control and flexibility, allowing for more complex iterator implementations beyond what generators can\noffer.\n\nUltimately, the choice between generators and iterators depends on the specific requirements of your program. Generators\nare often the preferred choice for simpler iterations and lazy evaluation, while iterators offer more customization and\ncontrol when dealing with complex iteration scenarios.\n\n### Magic methods\n\nMagic methods, also known as dunder methods, are special methods in Python that start and end with double underscores.\nThey are not meant to be invoked directly by the user, but are called internally by the class on certain actions.\nHere are some examples of commonly used magic methods in Python:\n\n`__new__(self)`: It is a static method that is called before the `__init__` method when creating an object. The primary purpose of **__new__** is to handle the object construction process and return an instance of the class.\n\n`__init__(self, ...)` : This method is called when an object is created and initialized. It takes arguments that are\nused to initialize the object's attributes.\n\n`__str__(self)` : This method is called when the object is printed as a string. It returns a string representation of\nthe object.\n\n`__len__(self)` : This method is called when the built-in **len()** function is called on the object. It returns the length\nof the object.\n\n`__add__(self, other)` : This method is called when the + operator is used on the object. It takes another object as an\nargument and returns the result of the addition.\n\n`__call__(self, other)`: The method in Python is a special method that allows an object to be called like a function\n\nHere's an example of a class that defines some of these magic methods:\n\n```\nclass Person:\n def __init__(self, name, age):\n self.name = name\n self.age = age\n\n def __str__(self):\n return f\"{self.name} is {self.age} years old.\"\n\n def __repr__(self):\n return f\"Person('{self.name}', {self.age})\"\n\n def __eq__(self, other):\n return self.name == other.name and self.age == other.age\n```\n\nHere is a simple example that demonstrates the difference between __str__ and __repr__:\n\n```angular2html\nclass Person:\n def __init__(self, name, age):\n self.name = name\n self.age = age\n\n def __str__(self):\n return f\"{self.name} ({self.age})\"\n\n def __repr__(self):\n return f\"Person(name='{self.name}', age={self.age})\"\n\nperson = Person(\"Alice\", 30)\nprint(str(person)) # Output: Alice (30)\nprint(repr(person)) # Output: Person(name='Alice', age=30)\n```\n\nIn this example, we define a Person class that has a **name** and an **age** attribute. We define both __str__ and __repr__ methods for the class. The __str__ method returns a string that is intended to be readable by humans, while the __repr__ method returns a string that is intended to be unambiguous and can be used to recreate the object.\nWhen we create a Person object and print it using the **str()** and **repr()** functions, we get different outputs. The **str()** function calls the __str__ method, which returns a user-friendly string representation of the object. The **repr()** function calls the __repr__ method, which returns a developer-friendly string representation of the object.\n\n### GIL\n\nThe **_Global Interpreter Lock_** is a mechanism in _CPython_ (the most common implementation of Python) that serves to serialize access to Python objects, _preventing multiple threads from executing Python bytecodes at once_.\nIn simple words, the GIL is a mutex (or a lock) that allows only one thread to hold the control of the Python interpreter.This means that _only one thread can be in a state of execution at any point in time_.\n\n### concurrency and parallelism\n\n**Concurrency** involves allowing multiple tasks to take turns accessing the same shared resources, like disk, network, or a\nsingle CPU core. **Parallelism**, on the other hand, is about maximizing the use of hardware resources, such as multiple CPU\ncores, to execute multiple tasks simultaneously. **Threading** is a way to achieve concurrency by running multiple threads\nwithin a single process, while **asyncio** is a way to achieve concurrency by running a single thread that can switch\nbetween multiple tasks.\n\n- **Threading**:\n\n The following code snippet demonstrates how to use threading to execute multiple tasks concurrently within a single\n process.\n\n```\nimport threading\n\ndef task1():\n # do some work\n\ndef task2():\n # do some work\n\n# create two threads to execute the tasks concurrently\nt1 = threading.Thread(target=task1)\nt2 = threading.Thread(target=task2)\n\n# start the threads\nt1.start()\nt2.start()\n\n# wait for the threads to finish\nt1.join()\nt2.join()\n```\n\n- **Asyncio**:\n\n The following code snippet demonstrates how to use asyncio to execute multiple tasks concurrently within a single\n thread:\n\n```\nimport asyncio\n\nasync def task1():\n # do some work\n\nasync def task2():\n # do some work\n\n# create an event loop\nloop = asyncio.get_event_loop()\n\n# execute the tasks concurrently within the event loop\nloop.run_until_complete(asyncio.gather(task1(), task2()))\n\n# close the event loop\nloop.close()\n\n```\n\n- **Multiprocessing**:\n\n The following code snippet demonstrates how to use multiprocessing to execute multiple tasks in parallel across\n multiple processes:\n\n```\nimport multiprocessing\n\ndef task1():\n # do some work\n\ndef task2():\n # do some work\n\n# create two processes to execute the tasks in parallel\np1 = multiprocessing.Process(target=task1)\np2 = multiprocessing.Process(target=task2)\n\n# start the processes\np1.start()\np2.start()\n\n# wait for the processes to finish\np1.join()\np2.join()\n\n```\n\nIn general, **concurrency** is preferred for **IO-bound** tasks, as it allows you to do something else while the IO resources\nare being fetched. **Parallelism**, on the other hand, is preferred for **CPU-bound** tasks, as it allows you to take advantage\nof multiple CPU cores to execute multiple tasks simultaneously.\n\n### Main types of methods in python classes\n\nIn Python, there are three main types of methods that can be defined in a class:\n\n1. **Instance Methods:**\n Instance methods are the most common type of methods used in Python classes. They are defined using the **self**\n parameter as the first argument. Instance methods can access and modify the instance attributes of the class.\n\n```\nclass MyClass:\n def my_instance_method(self, x, y):\n self.x = x\n self.y = y\n return self.x + self.y\n\nmy_object = MyClass()\nresult = my_object.my_instance_method(3, 4)\nprint(result) # Output: 7\n```\n\n2. **Class Methods**:\n Class methods are methods that are bound to the class and not the instance of the class. They are defined using the \\*\n **@classmethod** decorator, and they take the class itself as their first argument, typically named **cls**. Class\n methods can be called on the class itself, rather than on an instance of the class.\n\n```\nclass MyClass:\n class_attribute = 0\n\n @classmethod\n def my_class_method(cls, x, y):\n cls.class_attribute += 1\n return cls.class_attribute + x + y\n\nresult1 = MyClass.my_class_method(3, 4)\nprint(result1) # Output: 1 + 3 + 4 = 8\n\nresult2 = MyClass.my_class_method(1, 2)\nprint(result2) # Output: 2 + 1 + 2 = 5\n```\n\n3. **Static Methods:**\n Static methods are methods that don't depend on the class or instance. They are defined using the **@staticmethod**\n decorator and take no special first argument. Static methods are typically used for utility functions that don't\n require access to the class or instance.\n\n```\nclass MyClass:\n @staticmethod\n def my_static_method(x, y):\n return x + y\n\nresult = MyClass.my_static_method(3, 4)\nprint(result) # Output: 7\n```\n\n### Data serialization\n\nData serialization is the process of converting structured data into a format that allows sharing or storage of the data in a form that allows recovery of its original structure. In Python, there are several built-in modules for data serialization, including **pickle**, **json**, and **marshal**.\n**Note** that the marshal module is not suitable for serializing data that needs to be exchanged between different programming languages or platforms, as the binary format is specific to Python. For that purpose, you may want to consider using a different serialization library such as JSON, pickle, or protobuf.\n\nHere's an example of using the **pickle** module to serialize and deserialize a Python object:\n\n```\nimport pickle\n\n# Define a Python object\ngrades = {'Alice': 89, 'Bob': 72, 'Charles': 87}\n\n# Serialize the object to a byte stream\nserial_grades = pickle.dumps(grades)\n\n# Deserialize the byte stream back into a Python object\nreceived_grades = pickle.loads(serial_grades)\n\n# Print the original and received objects\nprint('Original object:', grades)\nprint('Serialized object:', serial_grades)\nprint('Received object:', received_grades)\n\n# Out put\n# Original object: {'Alice': 89, 'Bob': 72, 'Charles': 87}\n# Serialized object: b'\\x80\\x04\\x95#\\x00\\x00\\x00\\x00\\x00\\x00\\x00}\\x94(\\x8c\\x05Alice\\x94KY\\x8c\\x03Bob\\x94KH\\x8c\\x07Charles\\x94KWu.'\n# Received object: {'Alice': 89, 'Bob': 72, 'Charles': 87}\n\n```\n\nIn this example, we define a Python dictionary grades and serialize it using the **pickle.dumps()** function. We then deserialize the byte stream back into a Python object using the **pickle.loads()** function and print the original and received objects.\n\n### Data class in python\n\nOverall, data classes provide a convenient way to define classes that mainly hold data values, without having to write boilerplate code for initialization, representation, and comparisons.\nA **data class** is a class that is designed to only hold data values. It is similar to a regular class, but it usually doesn't have any other methods. It is typically used to store information that will be passed between different parts of a program or a system.\nHere's a simple example of a data class in Python:\n\n```angular2html\nfrom dataclasses import dataclass\n\n@dataclass\nclass Person:\n name: str\n age: int\n```\n\nIn the above example, the **@dataclass** decorator is used to create a data class called **Person**. The class has two attributes, name and age, which are defined using type annotations. The dataclass decorator automatically generates several special methods such as __init__(), __repr__(), and __eq__() for the class.\n\nTo add custom validation to the example using dataclasses, you can define a custom **__post_init__** method that runs after the object is initialized. Within this method, you can perform your custom validation logic. Here's how you can do it:\n\n```\nfrom dataclasses import dataclass\n\n@dataclass\nclass Person:\n name: str\n age: int\n\n def __post_init__(self):\n if self.age < 18:\n raise ValueError(\"Age must be greater than or equal to 18\")\n \n```\n\n### Shallow copy and deep copy\n\nIn Python, there are two types of copying: **shallow copy** and **deep copy**. A **shallow copy** creates a new object that stores references to the child objects of the original object. In contrast, a **deep copy** creates a new object that is completely independent of the original object.\nTo create a shallow copy of an object, we can use the copy method provided by the copy module in Python. The copy method returns a shallow copy of the object. For example:\n\n```angular2html\nimport copy\n\nlist1 = [1, 2, [3, 4]]\nlist2 = copy.copy(list1)\n\nprint(list1) # [1, 2, [3, 4]]\nprint(list2) # [1, 2, [3, 4]]\n\nlist2[2][0] = 5\n\nprint(list1) # [1, 2, [5, 4]]\nprint(list2) # [1, 2, [5, 4]]\n```\n\nIn this example, we create a shallow copy of **list1** using the copy method. When we modify the nested list in **list2**, the same change is reflected in **list1** because both lists share the same reference to the nested list.\nTo create a deep copy of an object, we can use the deepcopy method provided by the copy module. The deepcopy method returns a deep copy of the object. For example:\n\n```angular2html\nimport copy\n\nlist1 = [1, 2, [3, 4]]\nlist2 = copy.deepcopy(list1)\n\nprint(list1) # [1, 2, [3, 4]]\nprint(list2) # [1, 2, [3, 4]]\n\nlist2[2][0] = 5\n\nprint(list1) # [1, 2, [3, 4]]\nprint(list2) # [1, 2, [5, 4]]\n```\n\nIn this example, we create a deep copy of **list1** using the deepcopy method. When we modify the nested list in **list2**, the change is not reflected in **list1** because both lists have independent copies of the nested list.\n\n### Local and global variables\n\nIn summary, _local variables_ are declared inside a function or method and can only be accessed within that specific block, while _global variables_ are declared outside any function or method and can be accessed throughout the program and inside every function. To modify a global variable inside a function, we need to use the **global** keyword.\nHere are some simple examples of local and global variables in Python:\n\n```angular2html\n# Example of a local variable\ndef my_function():\n x = 5\n print(x)\n\nmy_function() # Output: 5\n\n# Example of a global variable\ny = 10\n\ndef my_function():\n print(y)\n\nmy_function() # Output: 10\n```\n\nIn this example, **x** is a local variable that is declared inside the **my_function** function and can only be accessed within that function. **y**, on the other hand, is a global variable that is declared outside any function and can be accessed inside the **my_function** function.\nTo modify a global variable inside a function, we need to use the **global** keyword. Here's an example:\n\n```angular2html\n# Example of modifying a global variable inside a function\nz = 15\n\ndef my_function():\n global z\n z = 20\n\nmy_function()\nprint(z) # Output: 20\n```\n\n### Comprehension\n\nComprehension is a concise and efficient way to create lists, dictionaries, and sets in Python. Multiple and nested comprehensions can be used to create complex data structures in a single line of code.\nHere's a very simple example of list comprehension:\n\n```angular2html\n# Example of list comprehension\nmy_list = [x for x in range(10)]\nprint(my_list) # Output: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n```\n\nIn this example, we use list comprehension to create a list of numbers from 0 to 9. The list comprehension is enclosed in square brackets and consists of an expression x followed by a for loop that iterates over a range of numbers from 0 to 9.\n\nHere's a very simple example of nested list comprehension:\n\n```angular2html\n# Example of nested list comprehension\nmy_list = [[x*y for y in range(5)] for x in range(5)]\nprint(my_list) # Output: [[0, 0, 0, 0, 0], [0, 1, 2, 3, 4], [0, 2, 4, 6, 8], [0, 3, 6, 9, 12], [0, 4, 8, 12, 16]]\n```\n\nIn this example, we use nested list comprehension to create a list of multiplication tables for numbers from 0 to 4. The outer list comprehension iterates over numbers from 0 to 4, and the inner list comprehension iterates over numbers from 0 to 4 to create a list of products.\n\n### Pydantic\n\nPydantic is a Python package that provides data validation and settings management using Python type annotations. It is a lightweight and flexible package that can be used to validate and parse data from various sources such as JSON, YAML, and databases.\nHere's a simple example of using Pydantic to define a data model and validate input data:\n\n```angular2html\nfrom pydantic import BaseModel\n\nclass Person(BaseModel):\n name: str\n age: int\n\nperson_data = {\n \"name\": \"John Doe\",\n \"age\": 30\n}\n\nperson = Person(**person_data)\nprint(person)\n\n\n```\n\nTo add custom validation to the example using Pydantic, you can use Pydantic's validation decorators like @validator. These decorators allow you to define custom validation functions that run when the model is being initialized. Here's how you can add custom validation to the Person model to ensure that the age is greater than or equal to 18:\n\n```\nfrom pydantic import BaseModel, validator\n\nclass Person(BaseModel):\n name: str\n age: int\n\n @validator(\"age\")\n def validate_age(cls, value):\n if value < 18:\n raise ValueError(\"Age must be greater than or equal to 18\")\n return value\n\nperson_data = {\n \"name\": \"John Doe\",\n \"age\": 30\n}\n\n```\n\nIn this example, we define a custom validation function **validate_age** using the **@validator** decorator. This function checks if the age value is less than 18, and if so, raises a **ValueError** with a custom error message. If the validation passes, the function returns the value as it is.\n\nIn this example, we define a Pydantic data model Person that has two fields: name of type str and age of type int.\nPydantic automatically validates the input data against the data model and raises a **ValueError** if the data is invalid. If the data is valid, Pydantic creates a new Person object with the validated data and assigns it to the person variable. We then print the person object to verify that it was created correctly.\n\n- Pydantic vs Data Classes:\n\nIn summary, Pydantic is more suitable when you need robust data validation, type conversion, and serialization capabilities. It's often used in scenarios where data integrity and consistency are critical, such as in web APIs or data validation pipelines. On the other hand, data classes are a simpler and more lightweight choice for defining basic data structures or DTOs (Data Transfer Objects) where validation and serialization are not primary concerns. Your choice between the two depends on your specific requirements and the complexity of your data handling needs.\n\n### Args and Kwargs in Python\n\nIn Python, ***args** and ****kwargs** are used to pass a variable number of arguments to a function. They allow you to handle arbitrary numbers of positional and keyword arguments, respectively.\n__args__ is used to pass a variable number of positional arguments to a function. It allows you to pass any number of arguments to a function without explicitly specifying them in the function definition. The arguments passed using *args are collected into a tuple within the function.\n\n```angular2html\ndef print_args(*args):\n for arg in args:\n print(arg)\nprint_args('Hello', 'World', '!')\n```\n\nOutput:\n\n```angular2html\nHello\nWorld\n!\n```\n\n****kwargs** is used to pass a variable number of keyword arguments to a function. It allows you to pass key-value pairs as arguments to a function without explicitly specifying them in the function definition. The arguments passed using ****kwargs** are collected into a dictionary within the function.\nHere's a simple example:\n\n```angular2html\ndef print_kwargs(**kwargs):\n for key, value in kwargs.items():\n print(key, value)\nprint_kwargs(name='John', age=25, city='New York')\n```\n\nOutput:\n\n```angular2html\nname John\nage 25\ncity New York\n```\n\nIn this example, the **print_kwargs()** function takes any number of keyword arguments using ****kwargs**. The key-value pairs are then printed one by one using a loop.\nYou can also combine ***args** and ****kwargs** in a function definition to accept both positional and keyword arguments\n\n### Operator overloading\n\nOperator overloading in Python refers to the ability to redefine the behavior of built-in operators **(+, -, *, /, etc.)** for user-defined classes.\nThis feature provides flexibility and allows you to make your classes work with operators in a way that makes sense for your specific use case.\n\nHere's a simple example to illustrate operator overloading using the **+** operator:\n\n```angular2html\nclass Point:\n def __init__(self, x, y):\n self.x = x\n self.y = y\n \n def __add__(self, other):\n new_x = self.x + other.x\n new_y = self.y + other.y\n return Point(new_x, new_y)\n\npoint1 = Point(2, 3)\npoint2 = Point(4, 5)\nresult = point1 + point2\nprint(result.x, result.y) # Output: 6 8\n```\n\nWhen we use the \"+\" operator between point1 and point2, it calls the **__add__()** method behind the scenes, allowing us to perform the addition operation in a customized way. The result is a new Point object with the summed coordinates.\n\n### Recursive function\n\nA recursive function in Python is a function that calls itself during its execution. It is a powerful technique used to solve problems by breaking them down into smaller, more manageable subproblems. Here's a simple example of a recursive function to calculate the factorial of a number:\n\n```angular2html\ndef factorial(n):\n if n == 0 or n == 1:\n return 1\n else:\n return n * factorial(n - 1)\n\nresult = factorial(5)\nprint(result) # Output: 120\n\n```\n\nRecursive functions are particularly useful when solving problems that can be divided into smaller subproblems of the same nature. They provide an elegant and concise way to express repetitive computations and can simplify complex algorithms.\nHowever, it's important to ensure that recursive functions have proper termination conditions to avoid infinite recursion. Recursive functions can consume more memory and may have performance implications compared to iterative solutions, so they should be used judiciously based on the problem at hand.\n\n### Context manager\n\nIn Python, a context manager is an object that defines the methods **__enter__** and **__exit__**.\nIt allows you to manage resources, such as files or database connections, in a structured and efficient manner. The **with** statement is used to create a context, and the context manager ensures that the resources are properly set up and cleaned up, even if an exception occurs.\nWhen you use the context manager with the **with** statement, the **__enter__** method is called at the beginning of the block, and the **__exit__** method is called at the end, regardless of whether an exception occurred or not. This ensures that resources are properly managed and cleaned up.\n\nLet's consider a practical example of a context manager for working with files. We'll create a context manager that opens a file, performs some operations, and automatically closes the file when exiting the context.\n\n```\nclass FileManager:\n def __init__(self, filename, mode):\n self.filename = filename\n self.mode = mode\n self.file = None\n\n def __enter__(self):\n self.file = open(self.filename, self.mode)\n return self.file\n\n def __exit__(self, exc_type, exc_value, traceback):\n self.file.close()\n\n# Using the file manager context manager with the 'with' statement\nwith FileManager(\"example.txt\", \"w\") as file:\n file.write(\"Hello, World!\")\n\n# File is automatically closed outside the context\n\n```\n\nIn this example, the **FileManager** class is a context manager that takes a **filename** and a **mode** (such as 'r', 'w', or 'a') as parameters. In the **__enter__** method, it opens the file in the specified mode and returns the file object. This allows us to work with the file within the with block.\n\nThe **__exit__** method is responsible for closing the file when exiting the context. Regardless of whether an exception occurs or not, the **__exit__** method will be called, ensuring that the file is closed properly.\n\n### Python 3.11 over previous versions\n\nHere are some simple examples of the advantages of using Python 3.11 over previous versions:\n\n- Fine-grained error locations in tracebacks:\n When an error occurs, Python 3.11 will point to the exact expression that caused the error, instead of just the line. For example, if you have a syntax error in a long line of code, Python 3.11 will point to the exact part of the line that caused the error, making it easier to debug.\n- Task and exception groups that simplify working with asynchronous code:\n Task groups provide a cleaner syntax for running and monitoring asynchronous tasks. For example, you can use task groups to run multiple tasks concurrently and wait for them to complete before continuing.\n- Faster code execution:\n Python 3.11 is faster than previous versions, which means that your Python programs will run faster. For example, if you have a program that performs a lot of computations, you will notice a significant speed improvement in Python 3.11.\n- A new built-in library for working with TOML files:\n Python 3.11 includes a new built-in library for working with TOML files, which is a popular configuration file format. For example, you can use the \"toml\" module to read and write TOML files in your Python programs.\n\nThese are just a few simple examples of the advantages of using Python 3.11 over previous versions\n\n### Semaphores and Mutexes\n\nSemaphores and Mutexes are synchronization mechanisms used in concurrent programming to control access to shared resources and avoid race conditions. Here, I'll explain the concepts of Semaphore and Mutex in Python\n\n- Mutex (Mutual Exclusion):\n A Mutex is a synchronization primitive that allows only one thread to access a shared resource at a time. It ensures that only one thread can acquire the mutex and access the protected resource, while other threads must wait until the mutex is released.\n\nHere's a simple Python example using the **threading** module:\n\n```\nimport threading\n\n# Create a mutex\nmutex = threading.Lock()\n\nshared_variable = 0\n\ndef increment_shared_variable():\n global shared_variable\n with mutex:\n shared_variable += 1\n\n# Create multiple threads to increment the shared variable\nthreads = []\nfor _ in range(5):\n thread = threading.Thread(target=increment_shared_variable)\n threads.append(thread)\n thread.start()\n\nfor thread in threads:\n thread.join()\n\nprint(\"Shared variable:\", shared_variable)\n\n```\n\nIn this example, the **mutex** ensures that only one thread can execute the critical section (incrementing **shared_variable**) at a time, preventing **race conditions**.\n\n- Semaphore:\n A Semaphore is a synchronization primitive that allows a fixed number of threads to access a resource simultaneously. It maintains a count, and threads can acquire or release the semaphore based on this count.\n\nHere's a simple Python example using the **threading** module:\n\n```angular2html\nimport threading\n\n# Create a semaphore with a maximum of 2 permits\nsemaphore = threading.Semaphore(2)\n\ndef access_shared_resource(thread_id):\n semaphore.acquire()\n print(f\"Thread {thread_id} is accessing the shared resource.\")\n # Simulate some work\n threading.Event().wait()\n print(f\"Thread {thread_id} is releasing the shared resource.\")\n semaphore.release()\n\n# Create multiple threads to access the shared resource\nthreads = []\nfor i in range(5):\n thread = threading.Thread(target=access_shared_resource, args=(i,))\n threads.append(thread)\n thread.start()\n\nfor thread in threads:\n thread.join()\n\n```\n\nIn this example, the semaphore allows up to 2 threads to access the shared resource concurrently, while other threads will wait until a permit is released.\n\n### Python built-in functions\n\nHere are some advanced Python built-in functions with simple examples:\n\n- `abs()` - returns the absolute value of a number\n\n```angular2html\nprint(abs(-5)) # Output: 5\n```\n\n- `all()` - returns True if all elements in an iterable are true\n\n```angular2html\nprint(all([True, True, False])) # Output: False\n```\n\n- `any()` - returns True if any element in an iterable is true\n\n```angular2html\nprint(any([True, True, False])) # Output: True\n```\n\n- `bin()` - converts an integer to a binary string\n\n```angular2html\nprint(bin(10)) # Output: 0b1010\n```\n\n- `enumerate()` - returns an iterator that generates tuples containing indices and values from an iterable\n\n```angular2html\nfruits = ['apple', 'banana', 'cherry']\nfor index, value in enumerate(fruits):\n print(index, value)\n# Output:\n# 0 apple\n# 1 banana\n# 2 cherry\n```\n\n- `map()` - returns an iterator that generates the results of applying a function to the elements of an iterable\n\n```angular2html\ndef square(num):\n return num ** 2\n\nnumbers = [1, 2, 3, 4, 5]\nsquared_numbers = list(map(square, numbers))\nprint(squared_numbers) # Output: [1, 4, 9, 16, 25]\n```\n\n- `max()` - returns the largest element in an iterable or the largest of two or more arguments\n\n```angular2html\nprint(max([1, 2, 3])) # Output: 3\n```\n\n- `pow()` - returns the result of raising a number to a power\n\n```\nprint(pow(2, 3)) # Output: 8\n```\n\n- `reversed()` - returns an iterator that generates the elements of a sequence in reverse order\n\n```\nfruits = ['apple', 'banana', 'cherry']\nfor fruit in reversed(fruits):\n print(fruit)\n# Output:\n# cherry\n# banana\n# apple\n```\n\n- `round()` - rounds a number to a specified number of decimal places\n\n```\nprint(round(3.14159, 2)) # Output: 3.14\n```\n\n- `zip()` - returns an iterator that generates tuples by aggregating the elements of several iterables\n\n```\nfruits = ['apple', 'banana', 'cherry']\nprices = [1.0, 2.0, 3.0]\nfor fruit, price in zip(fruits, prices):\n print(fruit, price)\n# Output:\n# apple 1.0\n# banana 2.0\n# cherry 3.0\n```\n\n- `format()` - formats a string using replacement fields\n\n```\nname = 'Alice'\nage = 30\nprint('My name is {0} and I am {1} years old.'.format(name, age))\n# Output: My name is Alice and I am 30 years old.\n```\n\n- `frozenset()` - returns an immutable frozenset object initialized with elements from the given iterable\n\n```angular2html\nvowels = ('a', 'e', 'i', 'o', 'u')\nf_set = frozenset(vowels)\nprint(f_set) # Output: frozenset({'a', 'e', 'i', 'o', 'u'})\n```\n\n- `getattr()` - returns the value of a named attribute of an object\n\n```\nclass Person:\n def __init__(self, name, age):\n self.name = name\n self.age = age\n\nperson = Person('Alice', 30)\nprint(getattr(person, 'name')) # Output: 'Alice'\n```\n\n- `hash()` - returns the hash value of an object\n\n```angular2html\nprint(hash('hello')) # Output: -3557965013271865832\n```\n\n- `isinstance()` - returns True if an object is an instance of a specified class\n\n```angular2html\nclass Person:\n pass\n\nperson = Person()\nprint(isinstance(person, Person)) # Output: True\n```\n\n- `issubclass()` - returns True if a class is a subclass of a specified class\n\n```angular2html\nclass Animal:\n pass\n\nclass Dog(Animal):\n pass\n\nprint(issubclass(Dog, Animal)) # Output: True\n```\n\n- `setattr()` - sets the value of a named attribute of an object\n\n```\nclass Person:\n def __init__(self, name, age):\n self.name = name\n self.age = age\n\nperson = Person('Alice', 30)\nsetattr(person, 'age', 31)\nprint(person.age) # Output: 31\n```\n\n- `delattr()` - deletes a named attribute from an object\n\n```\nclass Person:\n name = 'Alice'\n\nperson = Person()\ndelattr(person, 'name')\nprint(hasattr(person, 'name')) # Output: False\n```\n\n- `open()` - opens a file and returns a file object\n\n```\nfile = open('example.txt', 'r')\nprint(file.read()) # Output: This is an example file.\nfile.close()\n```\n\n- `slice()` - returns a slice object\n\n```\nmy_list = [1, 2, 3, 4, 5]\nmy_slice = slice(1, 4)\nprint(my_list[my_slice]) # Output: [2, 3, 4]\n```\n\n### Type Hints and Type Checking\n\nType hints in Python allow you to specify the expected types of variables, function parameters, and return values. This improves code readability, enables better IDE support, and allows static type checking with tools like `mypy`.\n\n**Basic Type Hints:**\n\n```python\nfrom typing import List, Dict, Optional, Union, Tuple\n\n# Function with type hints\ndef greet(name: str, age: int) -> str:\n return f\"Hello, {name}. You are {age} years old.\"\n\n# Variables with type hints\ncount: int = 10\nname: str = \"Alice\"\nis_active: bool = True\n\n# Collections\nnumbers: List[int] = [1, 2, 3, 4, 5]\nuser_data: Dict[str, Union[str, int]] = {\"name\": \"John\", \"age\": 30}\n```\n\n**Optional and Union Types:**\n\n```python\nfrom typing import Optional, Union\n\ndef find_user(user_id: int) -> Optional[Dict[str, str]]:\n # Returns Dict or None\n if user_id > 0:\n return {\"id\": str(user_id), \"name\": \"John\"}\n return None\n\ndef process_data(data: Union[str, int, float]) -> str:\n return str(data)\n```\n\n**Use Cases:**\n\n1. **Code Documentation:** Type hints serve as inline documentation\n2. **IDE Support:** Better autocomplete and error detection\n3. **Refactoring:** Safer refactoring with type checking\n4. **Team Collaboration:** Clearer code contracts between team members\n5. **Catching Bugs:** Static type checking can catch type-related errors before runtime\n\n### Package Management\n\nEffective package management is crucial for Python projects. Here are the main tools and practices:\n\n**pip and requirements.txt:**\n\n```python\n# requirements.txt\nDjango==4.2.0\nrequests==2.31.0\npytest==7.4.0\n\n# Install packages\n# pip install -r requirements.txt\n\n# Generate requirements.txt\n# pip freeze > requirements.txt\n```\n\n**Poetry (Modern Package Management):**\n\n```bash\n# Install Poetry\n# curl -sSL https://install.python-poetry.org | python3 -\n\n# Create a new project\npoetry new myproject\n\n# Add dependencies\npoetry add django\npoetry add pytest --dev\n\n# Install dependencies\npoetry install\n\n# Update dependencies\npoetry update\n```\n\n**pyproject.toml (Poetry configuration):**\n\n```toml\n[tool.poetry]\nname = \"myproject\"\nversion = \"0.1.0\"\ndescription = \"\"\n\n[tool.poetry.dependencies]\npython = \"^3.9\"\ndjango = \"^4.2.0\"\nrequests = \"^2.31.0\"\n\n[tool.poetry.dev-dependencies]\npytest = \"^7.4.0\"\nblack = \"^23.0.0\"\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n```\n\n**Virtual Environments:**\n\n```bash\n# Create virtual environment\npython -m venv venv\n\n# Activate (Windows)\nvenv\\Scripts\\activate\n\n# Activate (Linux/Mac)\nsource venv/bin/activate\n\n# Deactivate\ndeactivate\n```\n\n**Best Practices:**\n\n1. **Always use virtual environments** for project isolation\n2. **Pin dependency versions** in production (use `==` instead of `>=`)\n3. **Separate dev dependencies** from production dependencies\n4. **Use poetry or pip-tools** for better dependency resolution\n5. **Regularly update dependencies** to get security patches\n\n\n### functools Module\n\nThe `functools` module provides higher-order functions and operations on callable objects.\n\n\n**partial (Partial Application):**\n\n```python\nfrom functools import partial\n\ndef multiply(x, y, z):\n return x * y * z\n\n# Create a new function with some arguments pre-filled\nmultiply_by_2 = partial(multiply, 2)\nresult = multiply_by_2(3, 4) # Equivalent to multiply(2, 3, 4)\nprint(result) # Output: 24\n```\n\n\n**reduce (Functional Reduction):**\n\n```python\nfrom functools import reduce\n\nnumbers = [1, 2, 3, 4, 5]\nproduct = reduce(lambda x, y: x * y, numbers)\nprint(product) # Output: 120\n\n# Equivalent to:\nresult = 1\nfor num in numbers:\n result *= num\n```\n\n**Use Cases:**\n\n1. **Performance Optimization:** Cache expensive function calls\n2. **Function Composition:** Create specialized functions from general ones\n3. **Decorator Development:** Preserve function metadata in decorators\n4. **Functional Programming:** Apply functional patterns\n\n### Advanced Collections\n\nPython's `collections` module provides specialized container datatypes.\n\n**defaultdict (Default Dictionary):**\n\n```python\nfrom collections import defaultdict\n\n# Regular dict - raises KeyError for missing keys\nregular_dict = {}\n# regular_dict['missing'] # KeyError\n\n# defaultdict - returns default value\ndd = defaultdict(list)\ndd['fruits'].append('apple')\ndd['fruits'].append('banana')\nprint(dd['vegetables']) # Output: [] (empty list, not KeyError)\n```\n\n**Counter (Counting Elements):**\n\n```python\nfrom collections import Counter\n\n# Count occurrences\nwords = ['apple', 'banana', 'apple', 'cherry', 'banana', 'apple']\ncounter = Counter(words)\nprint(counter) # Output: Counter({'apple': 3, 'banana': 2, 'cherry': 1})\n\n# Most common\nprint(counter.most_common(2)) # Output: [('apple', 3), ('banana', 2)]\n```\n\n**OrderedDict (Remember Insertion Order):**\n\n```python\nfrom collections import OrderedDict\n\n# In Python 3.7+, regular dicts maintain order, but OrderedDict has extra features\nod = OrderedDict()\nod['first'] = 1\nod['second'] = 2\nod['third'] = 3\n\n# Move to end\nod.move_to_end('first')\nprint(list(od.keys())) # Output: ['second', 'third', 'first']\n```\n\n**deque (Double-Ended Queue):**\n\n```python\nfrom collections import deque\n\n# Efficient append/pop from both ends\ndq = deque([1, 2, 3])\ndq.appendleft(0) # Add to left\ndq.append(4) # Add to right\nprint(dq) # Output: deque([0, 1, 2, 3, 4])\n\ndq.popleft() # Remove from left\ndq.pop() # Remove from right\n```\n\n**ChainMap (Multiple Dictionaries):**\n\n```python\nfrom collections import ChainMap\n\ndict1 = {'a': 1, 'b': 2}\ndict2 = {'c': 3, 'd': 4}\ndict3 = {'b': 5, 'e': 6}\n\n# Chain maps - searches in order\nchain = ChainMap(dict1, dict2, dict3)\nprint(chain['b']) # Output: 2 (from dict1, first match)\nprint(chain['c']) # Output: 3 (from dict2)\n```\n\n**Use Cases:**\n\n1. **Data Grouping:** defaultdict for grouping operations\n2. **Frequency Analysis:** Counter for counting occurrences\n3. **Queue Operations:** deque for efficient queue/stack operations\n4. **Configuration Management:** ChainMap for layered configurations\n\n### Enum and NamedTuple\n\n**Enum (Enumerations):**\n\n```python\nfrom enum import Enum, auto\n\nclass Color(Enum):\n RED = 1\n GREEN = 2\n BLUE = 3\n\n# Usage\nprint(Color.RED) # Output: Color.RED\nprint(Color.RED.value) # Output: 1\nprint(Color.RED.name) # Output: RED\n\n# Auto values\nclass Status(Enum):\n PENDING = auto()\n PROCESSING = auto()\n COMPLETED = auto()\n\n# String enum\nclass HttpMethod(str, Enum):\n GET = \"GET\"\n POST = \"POST\"\n PUT = \"PUT\"\n DELETE = \"DELETE\"\n```\n\n**NamedTuple (Tuple with Named Fields):**\n\n```python\nfrom collections import namedtuple\n\n# Define a named tuple\nPoint = namedtuple('Point', ['x', 'y'])\n\n# Create instances\np1 = Point(1, 2)\np2 = Point(x=3, y=4)\n\nprint(p1.x, p1.y) # Output: 1 2\nprint(p1[0]) # Output: 1 (still indexable)\n\n# With type hints (Python 3.6+)\nfrom typing import NamedTuple\n\nclass Point(NamedTuple):\n x: int\n y: int\n\np = Point(1, 2)\nprint(p.x, p.y) # Output: 1 2\n```\n\n**Use Cases:**\n\n1. **Constants:** Enum for fixed set of values\n2. **Status Codes:** Represent states with enums\n3. **Data Structures:** NamedTuple for simple data containers\n4. **API Design:** Use enums for method types, status codes\n\n### asyncio (Advanced)\n\nAdvanced asyncio patterns for complex asynchronous programming.\n\n**Async Context Managers:**\n\n```python\nimport asyncio\n\nclass AsyncContextManager:\n async def __aenter__(self):\n print(\"Entering async context\")\n return self\n \n async def __aexit__(self, exc_type, exc_val, exc_tb):\n print(\"Exiting async context\")\n return False\n\nasync def main():\n async with AsyncContextManager() as acm:\n print(\"Inside async context\")\n await asyncio.sleep(1)\n\nasyncio.run(main())\n```\n\n**Async Generators:**\n\n```python\nasync def async_range(n):\n for i in range(n):\n await asyncio.sleep(0.1)\n yield i\n\nasync def main():\n async for value in async_range(5):\n print(value)\n\nasyncio.run(main())\n```\n\n**Task Groups (Python 3.11+):**\n\n```python\nasync def fetch_data(url):\n await asyncio.sleep(0.1)\n return f\"Data from {url}\"\n\nasync def main():\n async with asyncio.TaskGroup() as tg:\n task1 = tg.create_task(fetch_data(\"url1\"))\n task2 = tg.create_task(fetch_data(\"url2\"))\n task3 = tg.create_task(fetch_data(\"url3\"))\n \n # All tasks completed\n print(task1.result(), task2.result(), task3.result())\n\nasyncio.run(main())\n```\n\n**Use Cases:**\n\n1. **Concurrent I/O:** Handle multiple network requests simultaneously\n2. **Streaming Data:** Process data streams asynchronously\n3. **Real-time Applications:** WebSockets, chat applications\n4. **Resource Management:** Async context managers for resources\n\n### datetime Module\n\nComprehensive date and time handling.\n\n**Basic Operations:**\n\n```python\nfrom datetime import datetime, date, time, timedelta\n\n# Current date and time\nnow = datetime.now()\nprint(now) # Output: 2024-01-15 10:30:45.123456\n\n# Create specific date/time\ndt = datetime(2024, 1, 15, 10, 30, 45)\nprint(dt.date()) # Output: 2024-01-15\nprint(dt.time()) # Output: 10:30:45\n\n# Formatting\nformatted = dt.strftime('%Y-%m-%d %H:%M:%S')\nprint(formatted) # Output: 2024-01-15 10:30:45\n\n# Parsing\nparsed = datetime.strptime('2024-01-15', '%Y-%m-%d')\nprint(parsed) # Output: 2024-01-15 00:00:00\n```\n\n**Time Arithmetic:**\n\n```python\nfrom datetime import datetime, timedelta\n\nnow = datetime.now()\n\n# Add/subtract time\nfuture = now + timedelta(days=7, hours=3)\npast = now - timedelta(weeks=2)\n\n# Difference\ndiff = future - past\nprint(diff.days) # Output: 21\n\n# Timezone aware\nfrom datetime import timezone\nutc_now = datetime.now(timezone.utc)\nprint(utc_now) # Output: 2024-01-15 10:30:45.123456+00:00\n```\n\n**Use Cases:**\n\n1. **Date Calculations:** Calculate deadlines, durations\n2. **Logging:** Timestamp events\n3. **Scheduling:** Schedule tasks and reminders\n4. **Data Analysis:** Time series data processing\n\n\n### Abstract Base Classes (ABC) Advanced\n\nAdvanced usage of ABCs for interface definition.\n\n**Multiple Abstract Methods:**\n\n```python\nfrom abc import ABC, abstractmethod\n\nclass Shape(ABC):\n @abstractmethod\n def area(self):\n pass\n \n @abstractmethod\n def perimeter(self):\n pass\n \n def describe(self):\n return f\"Shape with area {self.area()} and perimeter {self.perimeter()}\"\n\nclass Rectangle(Shape):\n def __init__(self, width, height):\n self.width = width\n self.height = height\n \n def area(self):\n return self.width * self.height\n \n def perimeter(self):\n return 2 * (self.width + self.height)\n\n# Can't instantiate Shape directly\n# shape = Shape() # TypeError\n\nrect = Rectangle(5, 3)\nprint(rect.describe())\n```\n\n**Abstract Properties:**\n\n```python\nfrom abc import ABC, abstractproperty\n\nclass Animal(ABC):\n @abstractproperty\n def sound(self):\n pass\n\nclass Dog(Animal):\n @property\n def sound(self):\n return \"Woof!\"\n\ndog = Dog()\nprint(dog.sound) # Output: Woof!\n```\n\n**Use Cases:**\n\n1. **Interface Definition:** Define contracts for subclasses\n2. **Framework Design:** Create extensible frameworks\n3. **Type Checking:** Enable static type checking\n4. **Documentation:** Document expected methods\n\n\n### JSON vs Pickle Comparison\n\nUnderstanding serialization options.\n\n**JSON (Text-based, Portable):**\n\n```python\nimport json\n\ndata = {'name': 'John', 'age': 30, 'scores': [85, 90, 95]}\n\n# Serialize\njson_str = json.dumps(data)\nprint(json_str) # Output: {\"name\": \"John\", \"age\": 30, \"scores\": [85, 90, 95]}\n\n# Deserialize\nloaded = json.loads(json_str)\nprint(loaded) # Output: {'name': 'John', 'age': 30, 'scores': [85, 90, 95]}\n\n# File operations\nwith open('data.json', 'w') as f:\n json.dump(data, f)\n\nwith open('data.json', 'r') as f:\n loaded = json.load(f)\n```\n\n**Pickle (Binary, Python-specific):**\n\n```python\nimport pickle\n\nclass CustomClass:\n def __init__(self, value):\n self.value = value\n\nobj = CustomClass(42)\n\n# Serialize\npickled = pickle.dumps(obj)\n\n# Deserialize\nunpickled = pickle.loads(pickled)\nprint(unpickled.value) # Output: 42\n\n# File operations\nwith open('data.pkl', 'wb') as f:\n pickle.dump(obj, f)\n\nwith open('data.pkl', 'rb') as f:\n loaded = pickle.load(f)\n```\n\n**Comparison:**\n\n| Feature | JSON | Pickle |\n|---------|------|--------|\n| Format | Text | Binary |\n| Python objects | Limited | All types |\n| Security | Safe | Can execute code |\n| Portability | Cross-language | Python only |\n| Speed | Slower | Faster |\n| File size | Larger | Smaller |\n\n**Use Cases:**\n\n1. **JSON:** API communication, configuration files, data exchange\n2. **Pickle:** Python-specific data, complex objects, temporary storage\n3. **Security:** Always prefer JSON for untrusted data\n4. **Performance:** Pickle for internal Python applications\n\n\n### Composition vs Inheritance\n\nChoosing between composition and inheritance.\n\n**Inheritance:**\n\n```python\nclass Animal:\n def make_sound(self):\n pass\n\nclass Dog(Animal):\n def make_sound(self):\n return \"Woof!\"\n\nclass Cat(Animal):\n def make_sound(self):\n return \"Meow!\"\n```\n\n**Composition:**\n\n```python\nclass Engine:\n def start(self):\n return \"Engine started\"\n\nclass Car:\n def __init__(self):\n self.engine = Engine() # Composition\n \n def start(self):\n return self.engine.start()\n\ncar = Car()\nprint(car.start()) # Output: Engine started\n```\n\n**When to Use:**\n\n- **Inheritance:** \"is-a\" relationship, code reuse, polymorphism\n- **Composition:** \"has-a\" relationship, flexibility, avoid deep hierarchies\n\n**Use Cases:**\n\n1. **Inheritance:** Model hierarchies, shared behavior\n2. **Composition:** Flexible design, avoid coupling\n3. **Mixins:** Combine both approaches\n4. **Design Patterns:** Strategy, Decorator patterns use composition\n\n\n## Django related topics:\n\n### Django signals\n\nDjango signals can be used to automate tasks and update data in your application without having to manually perform\nthose tasks or update the data.\nLet's say we have a Django application that allows users to place orders for products. We want to automatically update\nthe product quantity in the database whenever an order is placed. We can use Django signals to accomplish this.\n\nLet's say you want to send an email notification whenever a new user is registered in your Django application. You can use Django's built-in signals for this purpose.\n\n- **Import necessary modules:**\n\n```angular2html\nfrom django.db import models\nfrom django.contrib.auth.models import User\nfrom django.db.models.signals import post_save\nfrom django.dispatch import receiver\nfrom django.core.mail import send_mail\n\n```\n\n- **Create a signal handler function to send an email:**\n\n```\n@receiver(post_save, sender=User)\ndef send_registration_email(sender, instance, created, **kwargs):\n if created:\n subject = 'Welcome to My Website'\n message = 'Thank you for registering on our website.'\n from_email = 'noreply@example.com'\n recipient_list = [instance.email]\n\n send_mail(subject, message, from_email, recipient_list)\n\n```\n\n- **Connect the signal handler:**\n\nYou need to connect the signal handler to the **post_save** signal of the **User** model. You can do this in your Django app's **apps.py** or **models.py** file, or in a separate **signals.py** file:\n\n```angular2html\nfrom django.apps import AppConfig\n\nclass YourAppConfig(AppConfig):\n default_auto_field = 'django.db.models.BigAutoField'\n name = 'your_app'\n\n def ready(self):\n import your_app.signals # Import the signals module where you defined your signal handler\n\n```\n\nNow, whenever a new user is registered in your Django application, the **send_registration_email** function will be triggered, sending a welcome email to the user.\n\n**Use cases:**\n\n1. [X] **User Authentication:** Use signals to perform custom actions when a user is registered or when a user logs in. or Send a welcome email to a user upon registration.\n2. [X] **Database Changes:** Notify users or perform other actions when a certain condition is met, like a low stock alert for an e-commerce application.\n3. [X] **Signals for Models:** Trigger actions when a model is created, updated, or deleted.\n4. [X] **File Uploads:** Perform post-processing tasks on uploaded files. For example, generating thumbnails after an image is uploaded.\n5. [X] **Notification Systems:** Send notifications to users or administrators when certain events occur, such as a new comment on a post.\n6. [X] **Django Middleware:** For instance, you can create a signal that fires before or after middleware processes a request, allowing you to perform custom actions at different points in the request/response cycle.\n7. [X] **Internationalization (i18n) and Localization (l10n):** Use signals to switch the language or locale dynamically based on user preferences or specific events within your application\n\n### Django middleware\n\nDjango middleware provides a way to process **requests** and **responses** globally in your application. Middleware can be used\nto perform authentication, logging, modifying headers, and more.\nLet's say we have a Django application that needs to add a custom header to every HTTP response. We can use Django\nmiddleware to accomplish this.\nHere are the steps to use Django middleware:\n\n1. Create a new file called **middleware.py** in the same directory as your settings.py file.\n2. Define a class that extends the **MiddlewareMixin** class.\n Implement the **process_response** method to modify the response object.\n3. Add your middleware class to the **MIDDLEWARE** setting in your **settings.py** file.\n\nHere's an example implementation:\n\n```\nclass CustomHeaderMiddleware:\n def process_response(self, request, response):\n response['X-Custom-Header'] = 'Hello, World!'\n return response\n```\n\nIn this example, we define a new middleware class called **CustomHeaderMiddleware** that extends the MiddlewareMixin class.\nWe implement the **process_response** method to modify the response object by adding a custom header.\nFinally, we add our middleware class to the MIDDLEWARE setting in our **settings.py** file:\n\n```\nMIDDLEWARE = [\n # Other middleware classes...\n 'myapp.middleware.CustomHeaderMiddleware',\n]\n```\n\nNow, whenever an HTTP response is returned by our application, the **CustomHeaderMiddleware** class will modify the response\nby adding a custom header.\n\n**Use cases:**\n\n1. [X] **Authentication and Authorization:** Middleware can check if a user is authenticated and has the required permissions to access a particular view.\n2. [X] **Security Measures:** Implement security-related tasks such as preventing Cross-Site Request Forgery (CSRF) attacks by adding CSRF tokens to forms or setting security headers like HTTP Strict Transport Security (HSTS) headers.\n3. [X] **Request Transformation:** Modify the request data before it reaches the view. For example, parsing JSON data from the request body and making it available to the view as a Python dictionary.\n4. [X] **Throttling and Rate Limiting:** Implement rate limiting to prevent abuse or overuse of your API or services by limiting the number of requests a client can make within a certain time frame.\n5. [X] **CORS (Cross-Origin Resource Sharing):** Enforce security policies for allowing or denying cross-origin requests to your application's resources.\n6. [X] **Content Compression:** Compress responses to reduce bandwidth usage and improve page load times, especially for resources like HTML, CSS, and JavaScript files.\n\n### Django custom template tags\n\nCustom template tags can be used to perform complex operations on data or to create reusable components for your\ntemplates.\nHere's a simple example of how to create a custom template tag in Django:\n\n1. Create a new Python module in one of your Django app's templatetags directory. For example, if you have an app named\n **myapp**, you could create a new module named **myapp_tags.py** in the **myapp/templatetags** directory.\n2. Define a function in the module that takes a template context and any arguments you want to pass to the tag. The\n function should return the value you want to output in the template. For example, here's a function that takes a list\n of strings and returns the first item in the list:\n\n```\nfrom django import template\n\nregister = template.Library()\n\n@register.simple_tag\ndef first_item(my_list):\n if my_list:\n return my_list[0]\n return ''\n```\n\nIn this example, we're using the **@register.simple_tag** decorator to register the function as a simple template tag.\n\n1. In your template, load the custom tag library and use the new tag in your HTML code. To load the custom tag library,\n add **{% load myapp_tags %}** at the top of your template. Here's an example of how to use the first_item tag we\n defined earlier:\n\n```\n{% load myapp_tags %}\n\n
    \n {% for item in my_list %}\n
  • {% first_item item %}
    • \n {% endfor %}\n
\n```\n\nIn this example, we're using the **first_item** tag to output the first item in each list item in a loop.\n\nHere are some common use cases for template tags in Django:\n\n1. [ ] **Displaying Data from the Database:** Template tags are often used to fetch and display data from the database. You can use **{% for %}** loops to iterate through querysets and display records, or use **{% if %}** tags to conditionally show content based on database values.\n2. [ ] **Including Other Templates:** You can use the **{% include %}** tag to include other templates within your template. This is useful for reusing common components across multiple pages.\n3. [ ] **Conditional Rendering:** Template tags like **{% if %}** and **{% else %}** are used for conditional rendering. You can show different content based on certain conditions.\n4. [ ] **Loading External Scripts and Styles:** You can use the **{% load %}** tag to load custom template tags or template libraries that provide additional functionality.\n5. [ ] **Extending Base Templates:** Template inheritance allows you to create a base template and extend it in other templates. This is useful for maintaining a consistent layout across your site.\n6. [ ] **Setting Variables:** The **{% with %}** tag allows you to set variables within a template, making it easier to reuse values.\n7. [ ] **Handling Forms:** Template tags like **{% csrf_token %}** are used for including CSRF tokens in forms to ensure security. Additionally, you can use tags like **{% forloop %}** to iterate through form fields.\n\nThese are just a few examples of how template tags can be used in Django templates to add dynamic behavior and logic to your web pages.\n\n### Django permissions\n\nDjango provides a built-in permissions system that allows you to control access to views and models in your application.\nPermissions can be assigned to users or groups, and can be checked in your views or templates to determine whether a\nuser has the necessary permissions to perform certain actions.\nHere's a simple example of how to use Django permissions:\n\n1. First, you need to define the permissions for your models. You can do this by adding a permissions attribute to\n your **model's meta class**. For example, let's say you have a Book model and you want to define a permission called\n **can_edit_book**:\n\n```\nfrom django.db import models\nfrom django.contrib.auth.models import User\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.ForeignKey(User, on_delete=models.CASCADE)\n published_date = models.DateField()\n\n class Meta:\n permissions = [\n (\"can_edit_book\", \"Can edit book\"),\n ]\n```\n\nIn this example, we've added a permissions attribute to the Book model's meta class, and defined a single permission\nnamed **can_edit_book**.\n\n5. Next, you need to assign the permissions to users or groups. You can do this using Django's built-in admin interface,\n or programmatically in your code. For example, let's say we want to assign the **can_edit_book** permission to a group\n called Editors:\n\n```\nfrom django.contrib.auth.models import Group\n\neditors_group, created = Group.objects.get_or_create(name='Editors')\neditors_group.permissions.add('myapp.can_edit_book')\n\n```\n\nIn this example, we're using the Group model to get or create a group called Editors, and then adding the\n**myapp.can_edit_book** permission to the group.\n\n9. Finally, you can check the user's permissions in your views or templates to control access to certain actions. For\n example, let's say we have a view that allows users to edit a book:\n\n```\nfrom django.shortcuts import get_object_or_404, render\nfrom django.contrib.auth.decorators import login_required, permission_required\nfrom myapp.models import Book\n\n@login_required\n@permission_required('myapp.can_edit_book', raise_exception=True)\ndef edit_book(request, book_id):\n book = get_object_or_404(Book, id=book_id)\n # perform edit operation\n return render(request, 'book_edit.html', {'book': book})\n```\n\nIn this example, we're using the **permission_required** decorator to check if the user has the myapp.can_edit_book\npermission before allowing them to edit the book. If the user does not have the necessary permission, a **PermissionDenied**\nexception will be raised.\n\n**Use cases:**\n\n1. [X] **Access Control for Views and URLs:** For example, allow only authenticated users to access their profile page, while allowing administrators to access an admin dashboard.\n2. [X] **Admin Panel Permissions:** Define who can create, modify, or delete records in the admin interface for specific models.\n3. [X] **API Access Control:** Implement role-based access control (RBAC) for APIs, allowing different levels of access for different user roles.\n\n### Django custom user models\n\nDjango's built-in **User** model provides a lot of functionality for authentication and authorization, but sometimes you may\nneed to customize the user model to include additional fields or functionality. In such cases, you can create a custom\nuser model that inherits from Django's **AbstractBaseUser** or **AbstractUser** classes.\nHere's a simple example of how to create a custom user model in Django:\n\n- Create a new app for your custom user model. For example, let's say you want to create a custom user model for a blog\n app. You could create a new app called **blog_auth**.\n- Create a new model for your custom user model. For example, let's say you want to add a **bio field** to your user\n model:\n\n```\nfrom django.contrib.auth.models import AbstractBaseUser, BaseUserManager, PermissionsMixin\nfrom django.db import models\n\nclass BlogUser(AbstractBaseUser, PermissionsMixin):\n email = models.EmailField(unique=True)\n first_name = models.CharField(max_length=30, blank=True)\n last_name = models.CharField(max_length=30, blank=True)\n bio = models.TextField(blank=True)\n\n is_staff = models.BooleanField(default=False)\n is_active = models.BooleanField(default=True)\n\n USERNAME_FIELD = 'email'\n EMAIL_FIELD = 'email'\n\n def __str__(self):\n return self.email\n```\n\nIn this example, we've created a new model called **BlogUser** that inherits from **AbstractBaseUser** and **PermissionsMixin**.\nWe've also defined the necessary fields, including a custom bio field.\n\n- Update your project settings to use the custom user model. In your project's **settings.py** file, update the \\*\n **AUTH_USER_MODEL** setting to point to your new user model:\n\n```\nAUTH_USER_MODEL = 'blog_auth.BlogUser'\n```\n\n- Migrate your database to create the new user model table. Run the following commands in your terminal:\n\n```\npython manage.py makemigrations\npython manage.py migrate\n```\n\nThat's it! You now have a custom user model in Django that you can use for authentication and authorization in your blog\napp.\n\n### Django Custom Managers\n\nDjango provides a default manager for each model that allows you to interact with the database. However, sometimes you\nmay need to customize the default manager to add additional functionality or implement custom queries.\nHere's a simple example of how to create a custom manager in Django:\n\n- Let's say we have a model called **Book** in our app that represents books in a library. We want to create a custom\n manager that returns only the books that are currently available for borrowing.\n\n```\nfrom django.db import models\n\nclass AvailableBooksManager(models.Manager):\n def get_queryset(self):\n return super().get_queryset().filter(is_available=True)\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.CharField(max_length=100)\n is_available = models.BooleanField(default=True)\n\n objects = models.Manager() # default manager\n available_books = AvailableBooksManager() # custom manager\n```\n\nIn this example, we've created a custom manager called **AvailableBooksManager** that filters the books based on the\n**is_available** field. We've also added a **available_books** attribute to the Book model that uses the custom manager.\n\n- Now we can use the custom manager in our views or templates to get only the books that are currently available for\n borrowing. For example:\n\n```\nfrom django.shortcuts import render\nfrom myapp.models import Book\n\ndef available_books(request):\n books = Book.available_books.all()\n return render(request, 'available_books.html', {'books': books})\n```\n\nIn this example, we're using the **available_books** manager to get all the books that are currently available for\nborrowing, and passing them to the **available_books.html** template.\n\n**Use cases:**\n\n1. [X] **Filtering and Query Optimization:** For example, you can create a manager that filters out inactive or deleted records by default.\n2. [X] **Data Validation and Preprocessing:** For instance, you can create a manager that ensures all data entered is in uppercase.\n3. [X] **Complex Query Construction:** For example, you can create a manager that fetches the top-rated products based on user reviews.\n4. [X] **Pagination and Result Limiting:** For example, Create a manager that retrieves a specified number of records per page.\n\n### Django Custom validators\n\nDjango provides a set of built-in validators that can be used to validate form data and model fields. However, sometimes\nyou may need to create your own custom validators to validate data in a specific way. Here's a simple example of how to\ncreate a custom validator in Django:\n\n- Let's say we have a model called Book in our app that represents books in a library. We want to create a custom\n validator that checks whether the book's title starts with a capital letter.\n\n```\nfrom django.core.exceptions import ValidationError\n\ndef validate_title(value):\n if not value[0].isupper():\n raise ValidationError(\"Title must start with a capital letter.\")\n\nclass Book(models.Model):\n title = models.CharField(max_length=100, validators=[validate_title])\n author = models.CharField(max_length=100)\n```\n\nIn this example, we've created a custom validator called **validate_title** that checks whether the first character of the\ntitle is a capital letter. If the validation fails, a ValidationError is raised. We've also added the validate_title\nvalidator to the title field of the Book model.\nNow when a user tries to create a book with a title that doesn't start with a capital letter, the validation will fail\nand an error message will be displayed.\n\n**Use cases**\n\n1. [X] **Password Strength Validation:** Ensure that user passwords meet certain complexity requirements, such as a minimum length, the presence of uppercase and lowercase characters, and special characters.\n2. [X] **Age Verification:** Validate that a user's age is above a certain threshold for age-restricted content or actions.\n3. [X] **URL or Domain Validation:** Verify that a URL or domain meets specific criteria, such as being a valid domain name or belonging to an allowed list of domains.\n4. [X] **File Type Validation:** Ensure that uploaded files have valid file extensions or meet specific criteria.\n5. [X] **Consistency Checks:** Ensure consistency between related fields in a model. For example, checking that start dates are earlier than end dates.\n\n### Custom management commands\n\nCustom management commands allow you to add your own functionality to the **manage.py** command, making it easy to automate\ntasks and perform custom operations on your Django project.\n\nHere is a simple example to demonstrate how to create and use custom management commands in Django:\n\n- Create a new Django app:\n\n```\npython manage.py startapp myapp\n```\n\n- Create a new directory called **management** inside the app directory, and create another directory called **commands** inside the management directory:\n\n```\nmkdir myapp/management\nmkdir myapp/management/commands\n```\n\n- Create a new Python file inside the commands directory, and name it **mycommand.py**. This will be the file that\n contains the code for your custom management command:\n\n```\nfrom django.core.management.base import BaseCommand\n\nclass Command(BaseCommand):\n help = 'My custom management command'\n\n def handle(self, *args, **options):\n print('Hello from my custom management command!')\n```\n\nRegister the new command with Django by adding an empty** **__init__.py** file inside the management directory:\n\n```\ntouch myapp/management/__init__.py\n```\n\n- Test the new command by running it from the command line:\n\n```\npython manage.py mycommand\n```\n\nYou should see the message **\"Hello from my custom management command!\"** printed to the console.\n\n**Use cases:**\n\n1. [X] **Data Import/Export:** Export data from your database to different formats for backup or analysis.\n2. [X] **Database Maintenance:** Implement commands for database maintenance tasks, such as creating database backups, purging old records, or optimizing database tables.\n3. [X] **User Management:** Create custom commands for managing users, such as creating user accounts in bulk, resetting passwords, or deactivating inactive users.\n4. [X] **Content Population:** Populate your database with sample or test data for development and testing purposes using custom commands.\n5. [X] **Cache Management:** Develop commands to clear or refresh your cache to keep your application's data cache up to date.\n6. [X] **API Data Fetching:** Create commands to fetch data from external APIs, update your database with the retrieved data, and perform any necessary data transformations.\n7. [X] **Report Generation:** Generate and distribute reports as PDFs, Excel sheets, or other formats using custom management commands.\n\n### Django's Query API\n\nHere are some examples of how to use Django's Query API:\n\n- Retrieving all objects from a model:\n\n```angular2html\nfrom myapp.models import MyModel\n\nall_objects = MyModel.objects.all()\n```\n\nThis will retrieve all objects from the **MyModel** model and store them in the **all_objects** variable.\n\n- Filtering objects based on a condition:\n\n```angular2html\nfrom myapp.models import MyModel\n\nfiltered_objects = MyModel.objects.filter(attribute=value)\n```\n\nThis will retrieve all objects from the **MyModel** model where the attribute matches the value and store them in the filtered_objects variable.\n\n- Chaining filters:\n\n```angular2html\nfrom myapp.models import MyModel\n\nfiltered_objects = MyModel.objects.filter(attribute1=value1).filter(attribute2=value2)\n```\n\nThis will retrieve all objects from the **MyModel** model where attribute1 matches value1 and attribute2 matches value2 and store them in the **filtered_objects** variable.\n\n- Querying related fields:\n\n```angular2html\nfrom myapp.models import MyModel\n\nrelated_objects = MyModel.objects.filter(related_model__attribute=value)\n```\n\nThis will retrieve all objects from the **MyModel** model where the related model's attribute matches value and store them in the **related_objects** variable.\n\n- Ordering results:\n\n```angular2html\nfrom myapp.models import MyModel\n\nordered_objects = MyModel.objects.order_by('attribute')\n```\n\nThis will retrieve all objects from the **MyModel** model and order them by the attribute field in ascending order. You can also use the - sign to order in descending order.\n\n- Limiting results:\n\n```angular2html\nfrom myapp.models import MyModel\n\nlimited_objects = MyModel.objects.all()[:10]\n```\n\nThis will retrieve the first 10 objects from the **MyModel** model and store them in the **limited_objects** variable.\n\n- Using OR conditions:\n\n```angular2html\nfrom myapp.models import MyModel\nfrom django.db.models import Q\n\nobjects = MyModel.objects.filter(Q(attribute1=value1) | Q(attribute2=value2))\n```\n\nThis will retrieve all objects from the **MyModel** model where either attribute1 matches value1 or attribute2 matches value2 and store them in the objects variable.\n\n- Using LIKE conditions:\n\n```angular2html\nfrom myapp.models import MyModel\n\nobjects = MyModel.objects.filter(attribute__contains=value)\n```\n\nThis will retrieve all objects from the **MyModel** model where attribute contains value and store them in the objects variable.\n\n- Using NOT conditions:\n\n```angular2html\nfrom myapp.models import MyModel\n\nobjects = MyModel.objects.exclude(attribute=value)\n```\n\nThis will retrieve all objects from the **MyModel** model where attribute does not match value and store them in the objects variable.\n\n- Retrieving a single object:\n\n```angular2html\nfrom myapp.models import MyModel\n\nobject = MyModel.objects.get(id=value)\n```\n\nThis will retrieve a single object from the **MyModel** model where **id** matches value and store it in the object variable.\n\n- Updating objects:\n\n```angular2html\nfrom myapp.models import MyModel\n\nMyModel.objects.filter(attribute=value).update(attribute=new_value)\n```\n\nThis will update all objects from the **MyModel** model where attribute matches value and set attribute to new_value.\n\n### Custom query expressions or Custom Lookup\n\nCustom query expressions are a way to extend the Django query API with your own custom SQL expressions. Custom query\nexpressions can be useful when you need to perform queries that are not easily expressible using the standard query API.\n\nSuppose you have a model called Person that represents a **person** with a first name, last name, and age. You want to\nperform a query that returns all people whose first name starts with a given letter. Here's how you can create a custom\nquery expression to perform this query:\n\n```\nfrom django.db.models import Lookup\n\nclass StartsWith(Lookup):\n lookup_name = 'startswith'\n\n def as_sql(self, compiler, connection):\n lhs, lhs_params = self.process_lhs(compiler, connection)\n rhs, rhs_params = self.process_rhs(compiler, connection)\n params = lhs_params + [rhs_params[0] + '%']\n return f\"{lhs} LIKE %s\", params\n\nPerson.objects.filter(first_name__startswith='A')\n```\n\nIn this example, we define a custom lookup called **StartsWith** that extends the Lookup class. We set the lookup_name\nattribute to **'startswith'** to define the name of the lookup. We then define the **as_sql** method to generate the SQL\nexpression for the lookup.\n\nHere's another simple example of defining and using a custom lookup in Django:\n\n```angular2html\nfrom django.db import models\nfrom django.db.models import Lookup\n\n\nclass GreaterThanLookup(Lookup):\n lookup_name = 'gt'\n def as_sql(self, compiler, connection):\n lhs, lhs_params = self.process_lhs(compiler, connection)\n rhs, rhs_params = self.process_rhs(compiler, connection)\n params = lhs_params + rhs_params\n return f'{lhs} > {rhs}', params\n\n\nmodels.IntegerField.register_lookup(GreaterThanLookup)\nclass Product(models.Model):\n name = models.CharField(max_length=100)\n price = models.DecimalField(max_digits=8, decimal_places=2)\n```\n\nUsage of the custom lookup\n\n```angular2html\nexpensive_products = Product.objects.filter(price__gt=100.00)\n```\n\nIn this example, we define a custom lookup called **GreaterThanLookup** that performs a greater-than comparison (>) for IntegerField fields. We subclass **django.db.models.Lookup** and override the **as_sql()** method to define the SQL representation of the lookup.\nThe usage of the custom lookup is demonstrated, where we filter the Product objects based on the price field using the gt lookup.\n\n### Django Filterset\n\nDjango Filterset provides a simple and intuitive way to filter data in Django. It allows you to define filterset classes that specify the fields to filter on and provides a range of built-in filters that you can use to filter data. You can also define custom filters and combine filters to create more complex filtering logic.\nHere are some simple examples of using Django Filterset:\n\n- **Basic filtering:**\n\nTo filter a queryset using Django Filterset, you first need to define a filterset class that specifies the fields to filter on. For example:\n\n```angular2html\nimport django_filters\nfrom .models import Product\n\nclass ProductFilter(django_filters.FilterSet):\n class Meta:\n model = Product\n fields = ['name', 'price']\n```\n\nThis defines a filterset class that filters on the **name** and **price** fields of the Product model. You can then use this filterset class to filter a queryset:\n\n```angular2html\nfrom .filters import ProductFilter\n\ndef product_list(request):\n queryset = Product.objects.all()\n filterset = ProductFilter(request.GET, queryset=queryset)\n return render(request, 'product_list.html', {'filterset': filterset})\n```\n\nThis filters the queryset based on the parameters provided in the GET request.\n\n- **Custom filtering:**\n\nYou can also define custom filters that perform more complex filtering logic. For example:\n\n```angular2html\nimport django_filters\nfrom .models import Product\n\nclass ProductFilter(django_filters.FilterSet):\n min_price = django_filters.NumberFilter(field_name='price', lookup_expr='gte')\n max_price = django_filters.NumberFilter(field_name='price', lookup_expr='lte')\n\n class Meta:\n model = Product\n fields = ['name', 'min_price', 'max_price']\n```\n\nThis defines custom filters that filter on the **price** field of the Product model. The **min_price** filter filters on products with a price greater than or equal to the provided value, and the **max_price** filter filters on products with a price less than or equal to the provided value.\nInstead of using FilterSets, you can directly use the Django ORM filters to perform filtering on your querysets. Django ORM provides a wide range of filter options such as exact **match, case-insensitive match, range filters**, and more. You can chain multiple filters together to create complex queries.\n\n**Use cases:**\n\n1. [X] **Search Functionality:** Implement a search feature allowing users to search for records based on specific criteria, such as keywords or partial matches in various fields.\n2. [X] **Data Filtering:** Create filters to allow users to filter data based on multiple fields, such as date ranges, categories, or status.\n3. [X] **Exporting Filtered Data:** Enable users to export filtered data to CSV, Excel, or other formats for further analysis or reporting.\n4. [X] **Geolocation Filtering:** Build Filtersets for location-based queries, such as finding nearby places or events within a specific radius.\n5. [X] **Data Visualization:** Integrate Filtersets with data visualization libraries to enable users to filter and explore data interactively.\n\n### Context managers\n\nThe context manager is responsible for setting up the context, running the block of code, and then cleaning up the context when the block is exited, regardless of whether the block raised an exception or not. In Django, context managers can be used to control signals, transactions, and caching.\nHere are some simple examples of context managers in Django:\n\n- **Database transactions**:\n In Django, context managers can be used to control database transactions. For example:\n\n```angular2html\nfrom django.db import transaction\n\nwith transaction.atomic():\n # code that requires a transaction\n```\n\n- **Caching**:\n Context managers can be used to control caching in Django. For example:\n\n```angular2html\nfrom django.core.cache import cache\n\nclass CacheContext:\n def __init__(self, key, value):\n self.key = key\n self.value = value\n\n def __enter__(self):\n cache.set(self.key, self.value)\n\n def __exit__(self, exc_type, exc_val, exc_tb):\n cache.delete(self.key)\n\nwith CacheContext('my_key', 'my_value'):\n # code that requires the cache to be set\n```\n\nIn summary, context managers in Django provide a simple and intuitive API for a powerful construct. They allow you to allocate and release resources precisely when you want to, and they make it easier to write safe and readable code.\nOverall, context managers provide a robust and efficient way to manage resources and ensure proper cleanup in Django applications. They contribute to cleaner code, improved error handling, and better resource utilization.\n\n**Use Cases:**\n\n1. [ ] **Database Transactions:** Use context managers to manage database transactions. This ensures that database changes are committed if the code block executes without errors or rolled back in case of exceptions.\n2. [ ] **File Handling:** Open and close files safely using a context manager, ensuring that files are closed properly even if an exception occurs.\n3. [ ] **Resource Locking:** Implement resource locking to ensure exclusive access to a resource, preventing concurrent access by multiple threads or processes.\n4. [ ] **Database Connections:** Manage database connections and cursors safely, ensuring that connections are closed after use.\n\n### Django Channels\n\nDjango Channels is a package that allows Django to handle WebSockets and other non-HTTP protocols. It extends the built-in capabilities of Django, allowing Django projects to handle not only HTTP but also protocols that require long-running connections, such as WebSockets, MQTT (IoT), chatbots, radios, and other real-time applications.\nHere are some differences between Django Channels and Django's built-in HTTP capabilities:\n\nDjango Channels:\n\n- Allows Django projects to handle protocols other than HTTP, such as WebSockets, MQTT, and chatbots\n- Provides support for Django's core features like authentication and sessions\n- Preserves the synchronous behavior of Django and adds a layer of asynchronous protocols allowing users to write views that are entirely synchronous, asynchronous, or a mixture of both\n- Replaces Django's default WSGI with its ASGI (Asynchronous Server Gateway Interface)\n\nDjango's built-in HTTP capabilities:\n\n- Handle only HTTP requests\n- Do not support protocols that require long-running connections, such as WebSockets and MQTT\n- Do not provide support for asynchronous code execution\n\nIn summary, Django Channels extends Django's built-in capabilities to handle protocols other than HTTP and provides support for asynchronous code execution.\n\n**Use Cases:**\n\n1. [X] **Real-Time Chat Applications:** Create real-time chat applications where users can send and receive messages instantly using WebSockets.\n2. [X] **Live Notifications:** Implement live notifications to inform users about new messages, friend requests, updates, or any other events that require immediate user attention.\n3. [X] **Multiplayer Online Games:** Develop real-time multiplayer games using WebSockets for game state synchronization, player interaction, and chat functionality.\n4. [X] **IoT Integration:** Integrate with Internet of Things (IoT) devices by handling real-time data streams, sensor data, and device control.\n5. [X] **Financial Trading Platforms:** Build financial trading applications that require real-time price updates, order execution, and live trading data.\n6. [X] **Geolocation and Mapping:** Develop applications that track and display the real-time location of vehicles, deliveries, or assets on a map.\n7. [X] **Live Video Streaming:** Build live video streaming platforms for webinars, conferences, or live events, where viewers can interact with the stream in real time.\n\n### HTTP methods in Django\n\nDjango provides built-in support for handling HTTP requests and responses using request and response objects Here are some examples of how to use HTTP methods in Django:\n\n- **GET**:\n\nTo handle a GET request in a class-based view, you can define a method called **get()** in your view class. For example:\n\n```angular2html\nfrom django.views import View\nfrom django.http import HttpResponse\n\nclass HelloView(View):\n def get(self, request):\n return HttpResponse('Hello, World!')\n```\n\n- **POST**:\n\nTo handle a POST request in a class-based view, you can define a method called **post()** in your view class. For example:\n\n```angular2html\nfrom django.views import View\nfrom django.http import HttpResponse\n\nclass LoginView(View):\n def post(self, request):\n username = request.POST['username']\n password = request.POST['password']\n # Authenticate user\n return HttpResponse('Logged in successfully')\n```\n\n- **PUT** and **DELETE**:\n\nDjango's class-based views do not have built-in support for handling PUT and DELETE requests. However, you can use mixins or third-party packages like Django REST framework to handle these requests. For example, using Django REST framework, you can define a class-based view that inherits from **APIView** and override the appropriate methods for PUT and DELETE requests:\n\n```angular2html\nfrom rest_framework.views import APIView\nfrom rest_framework.response import Response\n\nclass UserDetailView(APIView):\n def put(self, request, pk):\n # Update user object with pk\n return Response({'message': 'User updated'})\n\n def delete(self, request, pk):\n # Delete user object with pk\n return Response({'message': 'User deleted'})\n```\n\nIn Django, the difference between **PUT** and **PATCH** methods is similar to their difference in general HTTP terms.\n\n- **PUT** is used to modify an entire resource on the server. When a client sends a PUT request in Django, it updates the entire resource with the new data provided. If the resource does not exist, PUT creates a new resource. PUT is idempotent, meaning that calling it once or multiple times successively has the same effect.\n- **PATCH** is used to modify a part of a resource on the server. When a client sends a PATCH request in Django, it updates only the specified part of the resource with the new data provided. PATCH is not idempotent, meaning that successive identical PATCH requests may have additional effects. PATCH allows partial updates and side-effects on other resources.\n\nHere's an example of how to handle PUT and PATCH requests in Django using Django REST Framework:\n\n```angular2html\nfrom rest_framework.views import APIView\nfrom rest_framework.response import Response\nfrom rest_framework import status\nfrom myapp_models import MyResource\n\nclass MyResourceView(APIView):\n def put(self, request, pk):\n # Retrieve the resource with the given pk\n resource = MyResource.objects.get(pk=pk)\n\n # Update the entire resource with the new data\n resource.name = request.data.get('name')\n resource.age = request.data.get('age')\n resource.save()\n\n return Response(status=status.HTTP_200_OK)\n\n def patch(self, request, pk):\n # Retrieve the resource with the given pk\n resource = MyResource.objects.get(pk=pk)\n\n # Update only the specified part of the resource with the new data\n if 'name' in request.data:\n resource.name = request.data['name']\n if 'age' in request.data:\n resource.age = request.data['age']\n resource.save()\n\n return Response(status=status.HTTP_200_OK)\n```\n\nIn the above example, the put method is used to handle PUT requests and update the entire resource with the new name and age. The patch method is used to handle PATCH requests and update only the specified part of the resource. The request.data attribute is used to access the data sent in the request body.\n\n### annotate and aggregate in Django\n\n**annotate()** is typically used when you want to add additional information or calculated fields to each object in the queryset. For example, you can annotate a queryset of books with the number of authors for each book\nOn the other hand, **aggregate()** is used when you want to perform calculations on the entire queryset, such as calculating the sum, average, or count of a specific field across all objects in the queryset.\n\nThe output of **annotate()** is a QuerySet, which means it returns a modified queryset with the annotated values. On the other hand, **aggregate()** returns a dictionary containing the calculated aggregate values\n\nHere is a simple example of using annotate() and aggregate() in Django:\n\n```angular2html\nfrom django.db.models import Count\nfrom myapp.models import Book, Author\n\n# Count the number of books for each author\nauthors = Author.objects.annotate(num_books=Count('book'))\n\n# Count the total number of books\ntotal_books = Book.objects.aggregate(total=Count('id'))\n\n```\n\nIn the above example, **annotate()** is used to count the number of books for each author. The **Count()** function is used to count the number of related books for each author. The resulting queryset will have an additional attribute num_books for each author object.\n**aggregate()** is used to count the total number of books. The **Count()** function is used to count the number of books in the Book model. The resulting dictionary will have a single key total with the total number of books as its value.\nIt is important to note that **annotate()** returns a queryset, while **aggregate()** returns a dictionary.\n\n### Mixin in Django\n\nIn Django, a **mixin** is a class that provides additional functionality to other classes through **inheritance**. Mixins are used to add common or reusable functionality to multiple classes without the need for duplicating code.\nHere's a simple example to demonstrate how mixins work in Django:\n\n```angular2html\n# Define a mixin class\nclass TimestampMixin:\n created_at = models.DateTimeField(auto_now_add=True)\n updated_at = models.DateTimeField(auto_now=True)\n\n class Meta:\n abstract = True\n\n# Use the mixin in a model\nclass MyModel(TimestampMixin, models.Model):\n name = models.CharField(max_length=100)\n # other fields\n\n# Use the mixin in a view\nclass MyView(TimestampMixin, View):\n def get(self, request):\n # handle GET request\n pass\n\n def post(self, request):\n # handle POST request\n pass\n\n```\n\nIn this example, we have defined a mixin class called **TimestampMixin**. It adds two fields, **created_at** and **updated_at**, to any class that inherits from it. The Meta class with **abstract = True** ensures that the mixin itself is not treated as a model.\nBy using mixins, you can easily add common functionality to multiple classes without duplicating code. This promotes code reuse, improves maintainability, and keeps your codebase organized.\n\n**Use Cases:**\n\n1. [X] **Authentication Mixin:** Add authentication checks to views, ensuring that only authenticated users can access specific views.\n2. [X] **Permission Mixin:** Check user permissions or roles before allowing access to certain views, ensuring that users have the required privileges.\n3. [X] **Pagination Mixin:** Implement pagination for list views, allowing users to navigate through large datasets.\n4. [X] **Timestamp Mixin:** Add timestamp fields (created_at, updated_at) to models for automatic tracking of creation and modification times.\n5. [X] **Geolocation Mixin:** Add latitude and longitude fields to models, enabling geospatial queries and location-based services.\n6. [X] **Captcha Mixin:** Integrate CAPTCHA validation with forms to prevent spam submissions.\n7. [X] **File Upload Mixin:** Add file upload functionality to forms, allowing users to upload files like images or documents.\n8. [X] **User Tracking Mixin:** Implement middleware to track user activity, such as page views or interactions, and log them for analytics.\n\nThese are just a few examples of how mixins can enhance Django applications by promoting code reuse and modularity. They allow you to easily add and extend functionality across various components of your Django project, making your code more maintainable and efficient.\n\n### Cache in Django\n\nBy using caching in Django, you can significantly improve the performance of your application by reducing the load on the database and serving cached results faster. It is an essential technique for optimizing web applications and improving user experience.\nDjango provides built-in support for caching through its caching framework. The caching framework allows you to cache the results of views, template fragments, and even low-level database queries. It supports various cache backends and provides flexibility in configuring cache settings.\nHere's a simple example to demonstrate how caching works in Django:\n\n1- Configure the cache backend in your Django settings:\n\n```angular2html\nCACHES = {\n 'default': {\n 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',\n 'LOCATION': '127.0.0.1:11211',\n }\n}\n\n```\n\n2- Use the **cache_page** decorator to cache the result of a view:\n\n```angular2html\nfrom django.views.decorators.cache import cache_page\n\n@cache_page(60 * 15) # Cache the page for 15 minutes\ndef my_view(request):\n # Expensive computation or database query\n # ...\n return HttpResponse('Hello, World!')\n\n```\n\nWe then use the **cache_page** decorator to cache the result of the **my_view** function for 15 minutes. This means that the first time the view is accessed, the result will be computed and stored in the cache. For subsequent requests within the next 15 minutes, the cached result will be returned directly without executing the view function again.\n\n**Use Cases:**\n\n1. [X] **Page Caching:** Cache the rendered HTML of frequently accessed pages to reduce the load on the database and speed up page rendering.\n2. [X] **Queryset Caching:** Cache the results of database queries, especially for complex or frequently used queries. This reduces the need to hit the database repeatedly.\n3. [X] **Template Fragment Caching:** Cache specific portions of templates, such as navigation menus or sidebars, to minimize rendering time for frequently used components.\n4. [X] **Computed Values Caching:** Cache the results of expensive computations or data transformations to avoid recomputation for the same input.\n5. [X] **Third-Party API Responses:** Cache responses from third-party APIs to reduce the number of requests made to external services and improve the application's reliability.\n6. [X] **Rate Limiting:** Implement rate limiting by caching user request information to control the number of requests allowed within a specific time period.\n\nDjango provides a flexible caching framework that supports various caching backends, including in-memory caching, file-based caching, and distributed cache systems like Memcached and Redis. By strategically implementing caching in your Django application, you can significantly improve its performance and scalability.\n\n### Django constraint\n\nIn Django, a constraint refers to a rule or condition that can be applied to a database table to enforce data integrity. Constraints ensure that the data stored in the table follows certain rules and meets specific requirements.\nDjango provides a way to define constraints on model fields using the constraints attribute. This attribute allows you to specify one or more constraints for a model, such as unique constraints, check constraints, foreign key constraints, and more.\nHere's a simple example of defining and using a constraint in Django:\n\n```angular2html\nfrom django.db import models\nclass Person(models.Model):\n name = models.CharField(max_length=100)\n age = models.IntegerField()\n class Meta:\n constraints = [\n models.CheckConstraint(check=models.Q(age__gte=18), name='age_gte_18')\n ]\n```\n\nIn this example, we define a **Person** model with two fields: **name** and **age**. We want to enforce a constraint that ensures the age field is greater than or equal to 18.\nBy defining this constraint, Django will automatically create the necessary SQL statements to enforce the constraint when creating or modifying the database table for the Person model. Constraints help maintain data integrity and ensure that the data stored in the database follows the defined rules. They can be particularly useful in scenarios where you want to enforce specific conditions or relationships between fields in your models.\n\n- **Django constraints vs model validators differences**\n\nDjango constraints and model validators are two different mechanisms for ensuring data integrity in Django applications.\nConstraints are used to enforce data integrity rules at the database level, while validators are used to enforce data integrity rules at the application leve.\nDuring exceptions in Django Constraints raise database-level errors such as IntegrityError, while validators raise application-level errors such as ValidationError\n\n**Use Cases:**\n\n1. [X] **Unique Constraints:** Ensure that a specific field or combination of fields contains unique values across records. Commonly used for unique usernames, email addresses, or product codes.\n2. [X] **Check Constraints:** Specify custom validation rules for data in one or more fields. For example, ensure that a price field is always positive, or that a date falls within a certain range.\n3. [X] **Unique Together Constraints:** Ensure that a combination of fields contains unique values. Useful when you want to enforce uniqueness based on multiple fields, such as a combination of username and email.\n4. [X] **Table-Level Constraints:** Enforce constraints that involve multiple fields or records at the table level. For instance, ensuring that the start date is always earlier than the end date for date ranges.\n\n### bulk creation in Django\n\nIn Django, bulk creation refers to the process of creating multiple model instances in a single database query, rather than creating them one by one. This can significantly improve the performance of creating large numbers of objects. This method takes a list of model instances as an argument and inserts them into the database efficiently.\nHere's a simple example of using **bulk_create()** in Django:\n\n```angular2html\nfrom django.db import models\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.CharField(max_length=100)\n publication_year = models.IntegerField()\n\n\n# Create a list of Book instances\nbooks = [\n Book(title='Book 1', author='Author 1', publication_year=2020),\n Book(title='Book 2', author='Author 2', publication_year=2021),\n Book(title='Book 3', author='Author 3', publication_year=2022),\n]\n\n# Bulk create the books\nBook.objects.bulk_create(books)\n```\n\nIn this example, we define a **Book** model with three fields: title, author, and publication_year. We then create a list of Book instances with different values. To perform bulk creation, we use the **bulk_create()** method of the Book.objects manager. We pass the list of Book instances as an argument to **bulk_create()**, and Django will efficiently insert them into the database in a single query.\nOverall, **bulk_create()** is a useful feature in Django for efficiently creating multiple model instances in a single database query.\n\n**Use Cases:**\n\n1. [X] **Data Import and Migration:** Importing data from external sources, such as CSV files or JSON files, into your Django application's database.\n2. [X] **Bulk Updates:** Updating multiple rows in a database table simultaneously when changes need to be made to many records at once.\n3. [X] **Historical Data:** Maintaining historical records, such as version history, snapshots, or backups, by inserting historical data into dedicated tables.\n\n### prefetch_related and select_related in Django\n\nIn Django, **prefetch_related** and **select_related** are query optimization techniques that allow you to reduce the number of database queries when retrieving related objects.\n\n1- **select_related**\n\nIt works for **ForeignKey** and **OneToOneField** relationships. By using **select_related**, you can avoid the overhead of multiple database queries when accessing related objects.\nHere's a simple example:\n\n```angular2html\nfrom django.db import models\n\nclass Author(models.Model):\n name = models.CharField(max_length=100)\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.ForeignKey(Author, on_delete=models.CASCADE)\n```\n\nSuppose we have a **Book** model with a **ForeignKey** relationship to the **Author** model. If we want to retrieve all books and their corresponding authors, we can use **select_related** to fetch the related authors in a single query:\n\n```angular2html\nbooks = Book.objects.select_related('author').all()\n\nfor book in books:\n print(f\"Book: {book.title}, Author: {book.author.name}\")\n```\n\nIn this example, **select_related('author')** is used to fetch the related Author objects along with the Book objects in a single query. This avoids the need for an additional query for each book's author.\n\n2- **prefetch_related**\n\nIt works for **ManyToManyField** and **reverse ForeignKey** relationships. By using **prefetch_related**, you can reduce the number of queries when accessing related objects. Here's a simple example:\n\n```angular2html\nfrom django.db import models\n\nclass Category(models.Model):\n name = models.CharField(max_length=100)\n\nclass Product(models.Model):\n name = models.CharField(max_length=100)\n categories = models.ManyToManyField(Category)\n```\n\nSuppose we have a **Product** model with a **ManyToManyField** relationship to the **Category** model. If we want to retrieve all products and their corresponding categories, we can use **prefetch_related** to fetch the related categories efficiently:\n\n```angular2html\nproducts = Product.objects.prefetch_related('categories').all()\n\nfor product in products:\n print(f\"Product: {product.name}\")\n for category in product.categories.all():\n print(f\"Category: {category.name}\")\n```\n\nTo retrieve all authors along with their books efficiently, we can use **prefetch_related** as follows:\n\n```angular2html\nauthors = Author.objects.prefetch_related('books').all()\nfor author in authors:\n print(f\"Author: {author.name}\")\n for book in author.books.all():\n print(f\"Book: {book.title}\")\n```\n\nIn this example, **prefetch_related('books')** is used to fetch all the related Book objects efficiently for each author. It reduces the number of queries by fetching all the related books in a separate query, rather than fetching them individually for each author.\n\nIn this example, **prefetch_related('categories')** is used to fetch the related Category objects efficiently. It reduces the number of queries by fetching all the related categories in a separate query, rather than fetching them individually for each product.\nUsing **select_related** and **prefetch_related** can significantly improve the performance of your Django queries by reducing the number of database queries. It is especially useful when dealing with large datasets or complex relationships between models.\n\n### Third-party packages in Django\n\nThere are several third-party packages that can greatly enhance your development experience and provide additional functionality. Here is a list of must-have third-party packages that are commonly used in Django projects:\n\n- **Django REST framework**: A powerful and flexible toolkit for building Web APIs.\n- **Django Crispy Forms**: Easily manage Django forms' layout using bootstrap styles.\n- **Django Debug Toolbar**: Provides a set of panels displaying various debug information about the current request/response.\n- **Django Celery**: Distributed task queue system for asynchronous processing.\n- **Django allauth**: User registration, authentication, and account management.\n- **Django Rest Auth**: Provides a set of REST API endpoints for user registration, authentication, and account management.\n- **Django Filter**: Simplifies the process of filtering querysets dynamically.\n- **Django Rest Framework Swagger**: Generates interactive API documentation using the OpenAPI standard.\n- **Django Environ**: Allows you to define environment variables for your Django project.\n\n### Property decorators\n\nThe @property decorator in Django is used to define a method that behaves like a model attribute. It allows you to call custom model methods as if they were normal model attributes. Here's a simple example to illustrate the concept:\n\n```python\nfrom django.db import models\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.CharField(max_length=50)\n price = models.DecimalField(max_digits=5, decimal_places=2)\n @property\n def price_in_euros(self):\n return f\"{self.price} EUR\"\n```\n\nIn this example, we define a **Book** model with fields for **title**, **author**, and **price**. We then define a **price_in_euros** method with the **@property** decorator. This method returns the price of the book in euros as a string.\n\nBy using the **@property** decorator, we can access the **price_in_euros** method as if it were a model attribute. For example, we can retrieve the price of a book in euros as follows:\n\n```angular2html\nbook = Book.objects.get(pk=1)\nprint(book.price_in_euros) # Output: \"19.99 EUR\"\n\n```\n\nIn Django models, a method with the @property decorator behaves like a model attribute, while a method without this decorator behaves like a normal method.\nHere's an example to illustrate the difference:\n\n```angular2html\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.CharField(max_length=50)\n price = models.DecimalField(max_digits=5, decimal_places=2)\n\n @property\n def price_in_euros(self):\n return f\"{self.price} EUR\"\n\n def calculate_discounted_price(self, discount):\n return self.price * (1 - discount)\n\n```\n\nWhen we access **my_book.price_in_euros**, it behaves like a model attribute and returns the price of the book in euros as a string. On the other hand, when we call **my_book.calculate_discounted_price(0.1)**, it behaves like a normal method and returns the discounted price of the book.\n\n### WSGI and ASGI\n\nThe Web Server Gateway Interface (**WSGI**) and the Asynchronous Server Gateway Interface (**ASGI**) are two different specifications in Django for handling web requests and responses. Here's a brief explanation of each:\n\n#### WSGI (Web Server Gateway Interface):\n\n- WSGI is a synchronous interface that defines how web servers communicate with Python web applications.\n- It provides a standard way for web servers to forward requests to Python web frameworks and applications.\n- WSGI servers handle one request at a time and block until the response is generated, making it suitable for traditional synchronous web applications.\n- WSGI is the older and more established interface, widely used by Python web frameworks and servers.\n\n#### ASGI (Asynchronous Server Gateway Interface):\n\n- ASGI is an asynchronous interface that extends the capabilities of WSGI to support asynchronous web applications.\n- It allows multiple, concurrent requests to be handled asynchronously, making it suitable for real-time applications, long-polling, and WebSockets.\n- ASGI servers can handle both synchronous and asynchronous applications, providing flexibility for developers.\n- ASGI is designed to be a superset of WSGI, meaning that WSGI applications can be run inside ASGI servers.\n\nIn summary, **WSGI** is a synchronous interface for handling web requests and responses, while **ASGI** is an asynchronous interface that extends the capabilities of WSGI to support asynchronous applications.\n**ASGI** provides the ability to handle multiple concurrent requests and is suitable for real-time and long-polling applications.\n\n### Advanced features in ORM\n\nDjango ORM (Object-Relational Mapping) provides several advanced features that allow for more complex and powerful database queries and data manipulation. Here are some of the advanced features of Django ORM:\n\n- **Q objects:**\n\nQ objects allow for complex queries by combining multiple filters using logical operators like AND (&) and OR (|). They encapsulate keyword arguments for filtering and provide more flexibility in query construction\n\n```angular2html\nfrom django.db.models import Q\n\n# Query to retrieve books with either 'fiction' or 'mystery' genre\nbooks = Book.objects.filter(Q(genre='fiction') | Q(genre='mystery'))\n```\n\n- **F expressions:**\n\nF expressions allow you to perform database operations using field values without pulling the values from the database. They enable calculations and comparisons directly within the database query, improving performance and reducing round trips to the database\n\n```angular2html\nfrom django.db.models import F\n\n# Query to update the price of all books by increasing it by 10%\nBook.objects.all().update(price=F('price') * 1.1)\n```\n\n- **QuerySet methods**\n\nDjango provides a comprehensive set of QuerySet methods for retrieving, filtering, and manipulating data from the database. These methods include **annotate()**, **aggregate()**, **values()**, **distinct()**, **order_by()**, **reverse()**, and many more\n\n```angular2html\n# Query to retrieve the top 5 books with the highest ratings\ntop_books = Book.objects.order_by('-rating')[:5]\n\n# Query to retrieve the distinct authors of all books\nauthors = Book.objects.values('author').distinct()\n\n# Query to annotate the average price for each genre\ngenres = Book.objects.values('genre').annotate(avg_price=Avg('price'))\n```\n\n- **Model managers**\n\nModel managers allow you to customize the default QuerySet behavior by defining custom methods on the manager class. Managers provide a way to encapsulate common query logic and make it easily accessible across multiple models\n\n```angular2html\nclass BookManager(models.Manager):\n def get_best_sellers(self):\n return self.filter(sales__gt=1000)\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n author = models.CharField(max_length=50)\n price = models.DecimalField(max_digits=5, decimal_places=2)\n sales = models.IntegerField()\n\n objects = BookManager()\n\n# Query to retrieve the best-selling books\nbest_sellers = Book.objects.get_best_sellers()\n\n```\n\n- **Model inheritance**\n\nDjango supports model inheritance, allowing you to create more complex data models by building relationships between models. You can use abstract base classes, multi-table inheritance, and proxy models to implement different types of model inheritance\n\n```angular2html\nclass Publication(models.Model):\n title = models.CharField(max_length=100)\n publisher = models.CharField(max_length=50)\n\nclass Book(Publication):\n author = models.CharField(max_length=50)\n price = models.DecimalField(max_digits=5, decimal_places=2)\n\nclass Magazine(Publication):\n issue_number = models.IntegerField()\n\n# Query to retrieve all publications (books and magazines)\npublications = Publication.objects.all()\n\n```\n\nThese examples demonstrate the usage of advanced features in Django ORM, such as Q objects for complex queries, F expressions for database operations, QuerySet methods for data retrieval and manipulation, model managers for custom query logic, and model inheritance for creating relationships between models.\n\n### Class-based views methods\n\nDjango class-based views provide a set of methods that can be overridden to customize the behavior of the view. Here are some of the commonly used methods in Django class-based views:\n\n- **as_view()**:\n This method returns a callable view function that handles the request and generates the response. It is called when the URL pattern is matched to the view class.\n- **dispatch()**:\n This method is called by the **as_view()** method to handle the request. It inspects the request method (GET, POST, etc.) and calls the appropriate method (get(), post(), etc.) to generate the response.\n- **HTTP method handlers**:\n The HTTP method handlers (get(), post(), etc.) are called by the **dispatch()** method to generate the response. They perform the necessary operations (querying the database, rendering templates, etc.) and return an **HttpResponse** object.\n- **get_context_data()**:\n This method is called by the HTTP method handlers to retrieve the context data for the template. It returns a dictionary of context variables that are passed to the template.\n- **get(), post(), put(), delete()**:\n These methods are called by the **dispatch()** method to handle the corresponding HTTP methods. They perform the necessary operations (querying the database, rendering templates, etc.) and return an HttpResponse object.\n- **get_queryset()**:\n This method is used to retrieve the queryset that the view will use to retrieve objects. It can be overridden to customize the queryset based on the request.\n- **get_object()**:\n This method is used to retrieve a single object from the queryset. It can be overridden to customize the object retrieval based on the request.\n- **form_valid(), form_invalid()**:\n These methods are called when a form is submitted and validated. They perform the necessary operations (saving the form data, redirecting to a new page, etc.) and return an HttpResponse object.\n\nBy understanding these methods, you can customize the behavior of Django class-based views to suit your specific requirements and optimize the performance of your Django applications.\n\n### Django optimization\n\nTo optimize your Django application for scalability, there are several techniques you can implement:\n\n- **Database Optimization:**\n\nDjango's database layer provides various ways to improve performance, such as using database indexes, optimizing queries, and reducing database round trips\nSuppose you have a Django model called **Book** with a field called **title**. To optimize the database query, you can add an index to the title field by adding **db_index=True** to the field definition in the model:\n\n```angular2html\nclass Book(models.Model):\n title = models.CharField(max_length=100, db_index=True)\n author = models.CharField(max_length=50)\n \n```\n\nthis index will speed up queries that filter or order by the title field, as the database can use the index to quickly find the relevant rows\n\n- **Caching:**\n\nImplementing caching can significantly improve the performance of your Django application. You can use tools like Django's built-in caching framework. For example, to cache the result of a query for 5 minutes, you can use the **cache_page** decorator:\n\n```angular2html\nfrom django.views.decorators.cache import cache_page\n\n@cache_page(60 * 5)\ndef my_view(request):\n # perform database query\n books = Book.objects.all()\n \n```\n\n- **Code Optimization:**\n\nOptimizing your code can help improve the overall performance of your application. This includes writing efficient algorithms, reducing unnecessary database queries, and optimizing resource-intensive operations.\n\nSuppose you have a Django view that returns a list of all **Book** objects in the database. To optimize the code, you can use the **prefetch_related** method to fetch related objects in a single query:\n\ncode example:\n\n```angular2html\ndef book_list(request):\n # perform database query for all books\n books = Book.objects.all()\n # perform database query for each author of each book\n for book in books:\n authors = book.author_set.all()\n ...\n```\n\nOptimized version:\n\n```angular2html\ndef book_list(request):\n # fetch related authors in a single query\n books = Book.objects.prefetch_related('author_set').all()\n for book in books:\n authors = book.author_set.all()\n ...\n```\n\nIn the first example, the code performs a separate database query for each author of each book, resulting in a large number of queries. In the second example, the **prefetch_related** method fetches all related authors in a single query, reducing the number of queries and improving performance.\n\n- **Distributed Task Queues:**\n\nUsing distributed task queues, such as **Celery**, can help offload time-consuming tasks to separate worker processes, allowing your application to handle more concurrent requests.\nSuppose you have a view that performs a time-consuming task, such as sending an email. To offload the task to a separate worker process, you can use **Celery** such as below:\n\n```angular2html\nfrom celery import shared_task\n\n@shared_task\ndef send_email_task(email):\n # send email\n ...\n\ndef my_view(request):\n # offload task to Celery worker\n send_email_task.delay(email)\n ...\n\n\n```\n\n- **Scaling Horizontally:**\n\nTo handle increased traffic and user load, you can scale your Django application horizontally by splitting it into individual services (SOA) and scaling them independently. This approach allows you to distribute the load and optimize each service individually.\nSuppose you have a Django application that handles both user authentication and payment processing. To scale the application horizontally, you can split it into two separate services:\n1- Authentication service: Handles user authentication and authorization.\n2- Payment service: Handles payment processing and billing.\n\n- **Benchmarking:**\n\nRegularly benchmarking and profiling your application can help identify performance bottlenecks and areas for improvement. Tools like **Django Debug Toolbar** and **Django Silk** can assist in analyzing and optimizing your code.\n\n### Generic Foreign Key in Django\n\nIn Django, a Generic Foreign Key is a feature that allows you to create a relationship between a model and any other model in the database without directly specifying the related model. This is useful when you have a model that needs to be related to multiple other models. Instead of creating separate foreign keys for each related model, you can use a generic foreign key to achieve this flexibility.\n\nIn this example, we will create a model called **Vote**, which can be used to represent votes on different types of content, such as **posts**, **comments**, or **images**.\n\n1- Define the models in **models.py**:\n\n```angular2html\nfrom django.contrib.contenttypes.fields import GenericForeignKey\nfrom django.contrib.contenttypes.models import ContentType\nfrom django.db import models\n\nclass Vote(models.Model):\n content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)\n object_id = models.PositiveIntegerField()\n content_object = GenericForeignKey('content_type', 'object_id')\n\n voter_name = models.CharField(max_length=100)\n vote_type = models.CharField(max_length=10) # e.g., 'upvote', 'downvote'\n\n def __str__(self):\n return f\"{self.voter_name} voted on {self.content_object} ({self.vote_type})\"\n\n```\n\n2- Create instances of the models:\nNow, you can create instances of the **Vote** model and associate them with any other models in the database. For example:\n\n```angular2html\nfrom django.contrib.contenttypes.models import ContentType\n\n# Create a post instance (just an example, you can have other models)\npost = Post.objects.create(title=\"My Post\", content=\"This is my post content\")\n\n# Create a comment instance (just an example, you can have other models)\ncomment = Comment.objects.create(text=\"Nice post!\")\n\n# Create a vote for the post\npost_vote = Vote.objects.create(\n content_object=post,\n voter_name=\"John Doe\",\n vote_type=\"upvote\"\n)\n\n# Create a vote for the comment\ncomment_vote = Vote.objects.create(\n content_object=comment,\n voter_name=\"Alice\",\n vote_type=\"downvote\"\n)\n```\n\nIn this example, we have created a Vote model with a generic foreign key, allowing it to be associated with different types of content (posts, comments, etc.).\nYou can now use the **Vote** model to track votes on various content types throughout your Django project.\n\n### Django custom exceptions\n\nIn Django, you can create custom exceptions to handle specific error scenarios in your application. Custom exceptions allow you to define your own exception classes that inherit from Python's built-in **Exception** class or any other existing exception class. By doing so, you can raise and catch these custom exceptions in your code to handle different types of errors more effectively.\nHere's a simple example of how to create and use custom exceptions in Django:\n\n1- Define the custom exception in a **exceptions.py** file (you can create this file in your app directory):\n\n```angular2html\nclass InvalidDataError(Exception):\n def __init__(self, message=\"Invalid data provided.\"):\n self.message = message\n super().__init__(self.message)\n\n```\n\n2- Raise the custom exception in your views or other parts of the code:\n\n```angular2html\nfrom django.shortcuts import render\nfrom .exceptions import InvalidDataError\n\ndef custom_exception_view(request):\n try:\n # Some logic to validate data\n data = request.POST.get('data')\n if not data:\n raise InvalidDataError(\"Data is missing!\")\n\n # Rest of the code for processing valid data\n return render(request, 'success.html', {'data': data})\n\n except InvalidDataError as e:\n # Handle the custom exception\n error_message = str(e)\n return render(request, 'error.html', {'error_message': error_message})\n\n\n```\n\nIn this example, we create a view function called **custom_exception_view**. Inside the view, we perform some validation on the data received in the POST request. If the data is missing or invalid, we raise the **InvalidDataError** custom exception with a specific error message.\n\nhere are some examples of Exception Handling in Django:\n\n- Catching exceptions in views:\n\n```\nfrom django.http import HttpResponseServerError\ndef my_view(request):\n try:\n # Code that may raise an exception\n # ...\n except SomeException as e:\n # Code to handle SomeException\n return HttpResponseServerError(\"An error occurred: {}\".format(str(e)))\n```\n\nIn this example, if SomeException is raised within the try block, the corresponding except block will be executed. It\nreturns an HTTP 500 response with a custom error message.\n\n- Handling database-related exceptions:\n\nWhen working with Django's ORM (Object-Relational Mapping) and database operations, you may encounter exceptions related\nto database errors.\n\n```\nfrom django.db import DatabaseError\n\ndef my_view(request):\n try:\n # Perform database operations\n # ...\n except DatabaseError as e:\n # Handle database-related exception\n return render(request, 'error.html', {'message': str(e)})\n```\n\nIn this example, if a DatabaseError occurs during database operations, you can render an error template with the\nexception message.\n\n### select_for_update in Django\n\nIn Django, **select_for_update** is a method that allows you to lock database rows for update during a database query. This is useful in scenarios where you want to prevent multiple transactions from simultaneously modifying the same rows, thus avoiding potential conflicts and ensuring data consistency.\nWhen you use **select_for_update**, it places a \"select for update\" lock on the selected rows in the database. Other transactions attempting to modify the same rows will have to wait until the lock is released. This feature is particularly important in situations where concurrent updates could lead to incorrect results or data integrity issues.\n\nAssume we have a model called **Book** that represents books in a library. We want to ensure that when a user borrows a book, the book's availability status is updated, and during this process, no other user can borrow the same book until the update is complete.\n\n1- Define the model in **models.py**:\n\n```angular2html\nfrom django.db import models\n\nclass Book(models.Model):\n title = models.CharField(max_length=100)\n is_available = models.BooleanField(default=True)\n\n def __str__(self):\n return self.title\n\n```\n\n2- Borrow the book using **select_for_update**:\nIn your view or business logic, use **select_for_update** to borrow the book and update its availability status:\n\n```angular2html\nfrom django.db import transaction\nfrom .models import Book\n\ndef borrow_book(book_id):\n try:\n with transaction.atomic():\n book = Book.objects.select_for_update().get(pk=book_id)\n if book.is_available:\n book.is_available = False\n book.save()\n return f\"Successfully borrowed {book.title}.\"\n else:\n return f\"Sorry, {book.title} is not available for borrowing.\"\n\n except Book.DoesNotExist:\n return \"Book not found.\"\n\n\n```\n\nIn this example, we use a context manager (**with transaction.atomic()**) to ensure that the update operation is atomic and handled within a single database transaction. The **select_for_update()** method is called on the **Book.objects** queryset to obtain a lock on the selected row for update.\n\nThis example demonstrates how to use **select_for_update** to lock a specific row in the database during an update operation to maintain data integrity and consistency in a concurrent environment.\n\n### Django model methods\n\nTo list all the methods available for models in Django, we can refer to the methods provided by the base Model class and other related classes.\nHere are some common methods available for models in Django:\n\n- ```save():``` :\n Saves the model instance to the database.\n- ```delete():```\n Deletes the model instance from the database.\n- ```full_clean():```\n Performs model validation, including field validation, and raises any validation errors.\n- ```clean_fields(): ```\n Performs field validation and raises any validation errors.\n- ```clean():```\n Performs model-specific validation and raises any validation errors.\n- ```validate_unique():```\n Validates the uniqueness of fields and raises any validation errors.\n- ```refresh_from_db():```\n Reloads the model instance's fields from the database.\n- ```get_FOO_display(): ```\n Returns the display value of a choices field, where FOO is the name of the field.\n- ```get_absolute_url():```\n Returns the URL for displaying the model instance.\n- ```get_field_name():```\n Returns the name of the field associated with a given database column name.\n- ```get_next_by_FOO():```\n Returns the next model instance by a specified field, where FOO is the name of the field.\n- ```get_previous_by_FOO():```\n Returns the previous model instance by a specified field, where FOO is the name of the field.\n- ```serializable_value():```\n Returns the value of a field as a serializable object.\n- ```to_json():```\n Returns a JSON representation of the model instance.\n- ```to_dict():```\n Returns a dictionary representation of the model instance.\n- ```queryset():```\n Is used to customize the initial queryset used in a model's manager\n- ```pre_save()``` and ```post_save()```:\n Are signals that are emitted before and after saving a model instance\n- ```all()```:\n Returns a QuerySet containing all objects of the model from the database\n- ```filter()```:\n Is used to retrieve a subset of objects from the database based on specified criteria\n\n### Parametric unit tests\n\nIn Django, parametric unit tests refer to the practice of executing the same test logic with different input parameters or test data. This allows you to write more concise and reusable tests by avoiding code duplication.\nHere's a simple example of a parametric unit test in Django:\n\n```angular2html\nfrom django.test import TestCase\n\ndef multiply_numbers(a, b):\n return a * b\n\nclass MathTestCase(TestCase):\n def test_multiply_numbers(self):\n test_cases = [\n (2, 3, 6), # (a, b, expected_result)\n (0, 5, 0),\n (-4, 2, -8),\n (10, -3, -30),\n ]\n\n for a, b, expected_result in test_cases:\n result = multiply_numbers(a, b)\n self.assertEqual(result, expected_result)\n```\n\nWhen running the test, Django's test runner will execute the **test_multiply_numbers** method for each test case, providing detailed feedback for each iteration.\n\nUsing parametric unit tests like this helps to reduce code duplication and makes it easier to add, modify, or remove test cases. It also provides a clear overview of which test cases pass or fail and enables you to test your code against a variety of scenarios.\n\n\n### Testing in Django\n\nComprehensive testing is essential for Django applications:\n\n**Test Client:**\n\n```python\nfrom django.test import TestCase, Client\nfrom django.urls import reverse\n\nclass ViewTestCase(TestCase):\n def setUp(self):\n self.client = Client()\n self.user = User.objects.create_user(\n username='testuser',\n password='testpass123'\n )\n \n def test_home_page(self):\n response = self.client.get(reverse('home'))\n self.assertEqual(response.status_code, 200)\n \n def test_login_required(self):\n response = self.client.get(reverse('protected'))\n self.assertRedirects(response, '/login/')\n```\n\n**Mocking External Services:**\n\n```python\nfrom unittest.mock import patch, Mock\nfrom django.test import TestCase\n\nclass EmailTestCase(TestCase):\n @patch('myapp.views.send_email')\n def test_send_notification(self, mock_send_email):\n mock_send_email.return_value = True\n response = self.client.post('/send-notification/')\n self.assertTrue(mock_send_email.called)\n```\n\n**Factory Boy (Test Data Generation):**\n\n```python\n# Install: pip install factory_boy\nimport factory\nfrom django.contrib.auth.models import User\n\nclass UserFactory(factory.django.DjangoModelFactory):\n class Meta:\n model = User\n \n username = factory.Sequence(lambda n: f\"user{n}\")\n email = factory.LazyAttribute(lambda obj: f\"{obj.username}@example.com\")\n\n# Usage in tests\nuser = UserFactory()\n```\n\n**Fixtures:**\n\n```python\n# fixtures/initial_data.json\n[\n {\n \"model\": \"myapp.book\",\n \"pk\": 1,\n \"fields\": {\n \"title\": \"Test Book\",\n \"author\": 1\n }\n }\n]\n\n# In tests\nclass BookTestCase(TestCase):\n fixtures = ['initial_data.json']\n```\n\n**Use Cases:**\n\n1. **Unit Testing:** Test individual components in isolation\n2. **Integration Testing:** Test component interactions\n3. **API Testing:** Test REST API endpoints\n4. **Performance Testing:** Test query performance and optimization\n\n### Security Best Practices in Django\n\nSecurity is paramount in web applications. Here are essential security practices:\n\n**CSRF Protection:**\n\n```python\nfrom django.views.decorators.csrf import csrf_exempt, csrf_protect\n\n# CSRF is enabled by default\n# To exempt (use carefully):\n@csrf_exempt\ndef my_view(request):\n pass\n```\n\n**SQL Injection Prevention:**\n\n```python\n# BAD - Vulnerable to SQL injection\nUser.objects.extra(where=[\"username = '%s'\" % username])\n\n# GOOD - Use parameterized queries\nUser.objects.filter(username=username)\n```\n\n**XSS Prevention:**\n\n```python\n# In templates, always use:\n{{ user_input|escape }}\n# or\n{% autoescape on %}\n {{ user_input }}\n{% endautoescape %}\n```\n\n**Password Hashing:**\n\n```python\nfrom django.contrib.auth.hashers import make_password, check_password\n\n# Hash password\nhashed = make_password('mypassword')\n\n# Check password\nis_valid = check_password('mypassword', hashed)\n```\n\n**Security Settings:**\n\n```python\n# settings.py\nSECURE_SSL_REDIRECT = True\nSESSION_COOKIE_SECURE = True\nCSRF_COOKIE_SECURE = True\nSECURE_BROWSER_XSS_FILTER = True\nSECURE_CONTENT_TYPE_NOSNIFF = True\nX_FRAME_OPTIONS = 'DENY'\n```\n\n**Use Cases:**\n\n1. **Authentication:** Secure user authentication and authorization\n2. **Data Protection:** Protect sensitive data from unauthorized access\n3. **Input Validation:** Prevent injection attacks\n4. **HTTPS Enforcement:** Ensure secure communication\n\n### Logging and Monitoring in Django\n\nEffective logging and monitoring help track application behavior:\n\n**Logging Configuration:**\n\n```python\n# settings.py\nLOGGING = {\n 'version': 1,\n 'disable_existing_loggers': False,\n 'formatters': {\n 'verbose': {\n 'format': '{levelname} {asctime} {module} {message}',\n 'style': '{',\n },\n },\n 'handlers': {\n 'file': {\n 'level': 'INFO',\n 'class': 'logging.FileHandler',\n 'filename': 'django.log',\n 'formatter': 'verbose',\n },\n 'console': {\n 'level': 'DEBUG',\n 'class': 'logging.StreamHandler',\n 'formatter': 'verbose',\n },\n },\n 'loggers': {\n 'django': {\n 'handlers': ['file', 'console'],\n 'level': 'INFO',\n 'propagate': True,\n },\n 'myapp': {\n 'handlers': ['file', 'console'],\n 'level': 'DEBUG',\n },\n },\n}\n```\n\n**Using Loggers:**\n\n```python\nimport logging\n\nlogger = logging.getLogger(__name__)\n\ndef my_view(request):\n logger.info('Processing request')\n try:\n # Your code\n logger.debug('Debug information')\n except Exception as e:\n logger.error(f'Error occurred: {e}', exc_info=True)\n```\n\n**Sentry Integration:**\n\n```python\n# Install: pip install sentry-sdk\nimport sentry_sdk\nfrom sentry_sdk.integrations.django import DjangoIntegration\n\nsentry_sdk.init(\n dsn=\"your-sentry-dsn\",\n integrations=[DjangoIntegration()],\n traces_sample_rate=1.0,\n send_default_pii=True\n)\n```\n\n**Use Cases:**\n\n1. **Error Tracking:** Monitor and debug production errors\n2. **Performance Monitoring:** Track slow queries and operations\n3. **Audit Logging:** Log important user actions\n4. **Debugging:** Detailed logs for troubleshooting\n\n## FastAPI related topics:\n\n### Dependency Injection in FastAPI\n\nIn FastAPI, dependency injection is a powerful feature that allows you to declare dependencies for your route handlers, making your code modular, testable, and reusable. Dependencies can be functions or classes that provide the necessary data or services required by a route. FastAPI automatically manages the injection of these dependencies when a request is processed.\nHere's a simple example of dependency injection in FastAPI:\n\n```\nfrom fastapi import FastAPI, Depends\n\napp = FastAPI()\n\n# Dependency function\ndef get_query_parameter(q: str = None):\n return q or \"No query parameter provided.\"\n\n# Route using the dependency\n@app.get(\"/items/\")\nasync def read_item(commons: str = Depends(get_query_parameter)):\n return {\"message\": commons}\n\n```\n\n**get_query_parameter** is a dependency function that takes a query parameter **q** and returns its value or a default message if no parameter is provided. **Depends** is used to declare the dependency in the route handler (read_item). FastAPI will automatically execute the **get_query_parameter** function and inject its result into the commons parameter of the route handler. The route handler then uses the value provided by the dependency.\n\n#### use cases\n\n1. **Authentication:** You can use dependency injection to handle authentication. For example, a dependency function could verify an API key, OAuth token, or JWT token and provide user information to the route handler.\n2. **Database Connections:** Dependency injection is useful for managing database connections. You can create a dependency that establishes a database connection and injects it into route handlers that require database access.\n3. **Configuration Settings:** Injecting configuration settings (e.g., API keys, environment variables) into your route handlers allows for easy configuration changes without modifying the route functions.\n4. **Logging:** Use dependency injection to inject a logging service into route handlers, enabling consistent logging across your application.\n5. **Caching:** Dependencies can be employed for caching mechanisms. For instance, a dependency can check if the requested data is already cached and return it without hitting the actual data source.\n6. **Request Validation:** You can create a dependency function to handle request validation, checking headers, parameters, and request bodies before the route handler is executed.\n7. **Authorization:** Dependency injection can be used to handle authorization logic. A dependency function could check whether the authenticated user has the required permissions for a specific operation.\n8. **Rate Limiting:** Implementing rate limiting is another use case. A dependency can check the rate limit for a user and prevent excessive requests.\n\n### Dependency Ordering\n\n**Dependency ordering** in FastAPI refers to the order in which dependencies are executed. FastAPI executes dependencies in a top-down order, meaning that dependencies listed first are executed before those listed later. Understanding dependency ordering is crucial when you have dependencies that depend on the results of other dependencies.\n\nHere's an example to illustrate dependency ordering in FastAPI:\n\n```\nfrom fastapi import FastAPI, Depends\n\napp = FastAPI()\n\n# First dependency\nasync def common_parameters(q: str = None, skip: int = 0, limit: int = 10):\n return {\"q\": q, \"skip\": skip, \"limit\": limit}\n\n# Second dependency that depends on the result of the first one\nasync def query_processing(params: dict = Depends(common_parameters)):\n # Some processing using the result of the first dependency\n processed_query = f\"Processed query: {params['q']}, Skip: {params['skip']}, Limit: {params['limit']}\"\n return {\"processed_query\": processed_query}\n\n# Route using the dependencies\n@app.get(\"/items/\")\nasync def read_item(query_result: dict = Depends(query_processing)):\n return query_result\n\n```\n\n**common_parameters** is the first dependency that provides common parameters like q, skip, and limit. **query_processing** is the second dependency that depends on the result of common_parameters. It performs some processing based on the parameters obtained from the first dependency. The route handler **read_item** depends on the result of **query_processing**. It uses the processed query result in the route response.\n\n### Pydantic methods in FastAPI\n\nIn FastAPI, models are typically defined using Pydantic, a data validation and parsing library. Pydantic models in FastAPI can utilize various methods to customize their behavior. These methods help in validating, parsing, and processing the data before it is used in your application. Let's explore some of these methods with examples and discuss their use cases in real-world applications.\n\n- @validator Decorator:\n You can use the @validator decorator to define custom validation logic for a specific field.\n\n```\nfrom typing import List\nfrom pydantic import BaseModel, validator\n\nclass Item(BaseModel):\n name: str\n price: float\n\n @validator(\"price\")\n def validate_price(cls, value):\n if value < 0:\n raise ValueError(\"Price must be non-negative\")\n return round(value, 2)\n\n```\n\n- @root_validator Decorator:\n Use Case: Use the @root_validator decorator to perform validation that involves multiple fields.\n\n```\nfrom pydantic import BaseModel, root_validator\n\nclass Item(BaseModel):\n name: str\n price: float\n quantity: int\n\n @root_validator\n def validate_total_price(cls, values):\n total_price = values['price'] * values['quantity']\n if total_price > 100:\n raise ValueError(\"Total price cannot exceed 100\")\n return values\n\n```\n\n### Decorators in FastAPI\n\nIn FastAPI, decorators are used to modify the behavior of functions or class methods. They are functions that take another function as an argument and extend or modify the behavior of that function. Decorators in FastAPI are often used for creating reusable components, handling dependencies, and adding extra functionality to route handlers or other parts of your application.\nLet's start with a simple example of using decorators in FastAPI:\n\n```\nfrom fastapi import FastAPI, Depends\n\napp = FastAPI()\n\n# Decorator to log information before and after handling a request\ndef log_request(func):\n async def wrapper(*args, **kwargs):\n print(\"Request received\")\n response = await func(*args, **kwargs)\n print(\"Request handled\")\n return response\n return wrapper\n\n# Using the decorator for a route\n@app.get(\"/items/\", dependencies=[Depends(log_request)])\nasync def read_items():\n return {\"message\": \"Items retrieved\"}\n\n```\n\n#### Real-World Use Cases of decorators:\n\n- **Authentication and Authorization**: To check if a user is authenticated before allowing access to certain routes.\n- **Rate Limiting**: To limit the rate of requests to a specific route.\n- **Logging and Monitoring**: To log information about incoming requests and responses.\n- **Validation and Transformation**: To validate input data before processing it.\n\n### WebSockets in FastAPI\n\nWebSocket is a communication protocol that provides full-duplex communication channels over a single, long-lived connection. FastAPI supports WebSocket communication, allowing you to build real-time applications, such as chat applications, live updates, notifications, or collaborative editing. Let's go through a simple example of using WebSocket in FastAPI and then discuss potential use cases.\n\nNow, let's create a simple WebSocket application:\n```\nfrom fastapi import FastAPI, WebSocket\n\napp = FastAPI()\n\n# WebSocket endpoint\n@app.websocket(\"/ws\")\nasync def websocket_endpoint(websocket: WebSocket):\n await websocket.accept()\n while True:\n data = await websocket.receive_text()\n await websocket.send_text(f\"Message text was: {data}\")\n\n```\nThe **/ws** route is designated as a WebSocket endpoint. The **websocket_endpoint** function is called when a WebSocket connection is established. It accepts the WebSocket connection and enters a loop to continuously receive and send messages.\n\nThe **await websocket.receive_text()** line waits for incoming messages, and **await websocket.send_text()** sends a response back to the client.\n\n#### Real-World Use Cases of websocket:\n- **Real-Time Chat Applications:** Implementing real-time chat applications where users can send and receive messages instantly.\n- **Live Updates and Notifications:** Providing live updates and notifications to users, such as news updates, sports scores, or social media notifications.\n- **Real-Time Monitoring and Dashboards:** Building real-time monitoring dashboards that display live data, such as system metrics, financial data, or IoT sensor readings.\n- **Financial Trading Platforms:** Developing financial trading platforms where real-time updates of stock prices, trades, and market data are crucial.\n- **Job Processing and Notifications:** Providing real-time updates on the progress of long-running tasks or job processing.\n\n### Asynchronous File Uploads\nFastAPI supports asynchronous file uploads, allowing you to handle large file uploads efficiently. This is especially beneficial when dealing with web applications that need to process file uploads without blocking the server for other requests. \n\nNow, let's create a FastAPI application with an asynchronous file upload endpoint:\n\n```angular2html\nfrom fastapi import FastAPI, File, UploadFile\n\napp = FastAPI()\n\n# Asynchronous file upload endpoint\n@app.post(\"/uploadfile/\")\nasync def create_upload_file(file: UploadFile = File(...)):\n return {\"filename\": file.filename}\n\n```\nThe **/uploadfile/** route is designated as an asynchronous file upload endpoint.\nThe **create_upload_file** function is called when a POST request is made to **/uploadfile/** with a file attached.\nThe **UploadFile** class from FastAPI's **File** module is used to handle file uploads asynchronously.\n\n#### Real-World Use Cases of Asynchronous File Uploads:\n- **Large File Uploads:** Allowing users to upload large files, such as images, videos, or documents, without causing timeouts or blocking other requests.\n- **Image and Video Processing:** Processing images or videos uploaded by users asynchronously, such as generating thumbnails, applying filters, or transcoding videos.\n- **Document Processing:** Handling document uploads, such as PDFs or text files, and processing them asynchronously, such as extracting text, generating previews, or converting formats.\n- **Audio File Processing:** Uploading audio files for processing, such as extracting metadata, analyzing content, or converting formats.\n\n### Security Headers\n\nSecurity headers play a crucial role in web applications to enhance security by preventing certain types of attacks. FastAPI allows you to set security headers easily, providing protection against common web vulnerabilities.\nNow, let's create a FastAPI application with custom security headers:\n\n```\nfrom fastapi import FastAPI\nfrom fastapi.middleware.trustedhost import TrustedHostMiddleware\n\napp = FastAPI()\n\n# Middleware for trusted hosts\napp.add_middleware(TrustedHostMiddleware, allowed_hosts=[\"example.com\", \"sub.example.com\"])\n\n# Middleware for security headers\n@app.middleware(\"http\")\nasync def add_security_headers(request, call_next):\n response = await call_next(request)\n\n # Set security headers\n response.headers[\"Strict-Transport-Security\"] = \"max-age=31536000; includeSubDomains\"\n response.headers[\"Content-Security-Policy\"] = \"default-src 'self'\"\n response.headers[\"X-Content-Type-Options\"] = \"nosniff\"\n response.headers[\"X-Frame-Options\"] = \"DENY\"\n response.headers[\"X-XSS-Protection\"] = \"1; mode=block\"\n response.headers[\"Referrer-Policy\"] = \"no-referrer\"\n\n return response\n\n# Route to demonstrate security headers\n@app.get(\"/\")\nasync def read_root():\n return {\"message\": \"Welcome to the secure endpoint!\"}\n\n```\nThe **TrustedHostMiddleware** is used to enforce a list of allowed hostnames for better protection against HTTP Host header attacks.\nThe **add_security_headers** function is a middleware that sets various security headers in the HTTP response.\n\nThe security headers set include:\n\n- **Strict-Transport-Security (HSTS):** Enforces the use of HTTPS for a specified duration.\n- **Content-Security-Policy (CSP):** Defines content security policies to mitigate XSS attacks.\n- **X-Content-Type-Options:** Prevents browsers from MIME-sniffing a response.\n- **X-Frame-Options:** Prevents the browser from rendering a page in a frame.\n- **X-XSS-Protection:** Enables a browser's built-in XSS protection.\n- **Referrer-Policy:** Controls how much information is included in the Referer header.\n\nImplementing these security headers is an essential part of securing web applications and protecting against a wide range of common web vulnerabilities. They contribute to creating a robust and secure environment for your users and help in maintaining compliance with security standards and best practices.\n\n### Background Tasks\nBackground tasks in FastAPI allow you to execute functions asynchronously in the background, separate from the main request-response cycle. This is particularly useful for tasks that do not need an immediate response but can be performed in the background to improve the overall responsiveness of your application. \nNow, let's create a FastAPI application with a background task:\n\n```angular2html\nfrom fastapi import FastAPI, BackgroundTasks\nimport time\n\napp = FastAPI()\n\n# Background task\ndef process_data(background_tasks: BackgroundTasks, data: str):\n # Simulate a time-consuming task\n time.sleep(5)\n print(f\"Processing data: {data}\")\n\n# Route using the background task\n@app.post(\"/process_data/\")\nasync def create_process_data(data: str, background_tasks: BackgroundTasks):\n background_tasks.add_task(process_data, data)\n return {\"message\": \"Data processing started in the background\"}\n\n```\nThe **process_data** function is a background task that simulates a time-consuming operation. In a real-world scenario, this could be a task like sending emails, processing images, updating records in a database, etc.\n\nThe **create_process_data** route uses the **BackgroundTasks** dependency to add the **process_data** background task. The route returns a response immediately, and the background task runs asynchronously.\n\n### Middleware in FastAPI\nMiddleware in FastAPI is a function that works with every request before it is processed by any specific path operation and with every response before returning it. \nYou can add middleware to FastAPI applications using the **@app.middleware(\"http\")** decorator on top of a function. The middleware function receives the request and a **call_next** function that will receive the request as a parameter. It then passes the request to be processed by the rest of the application and takes the response generated by the application, allowing you to modify it further before returning it\n\nHere's a simple example of a middleware that adds a custom header to the response to measure the processing time:\n\n```\nfrom fastapi import FastAPI, Request\nimport time\n\napp = FastAPI()\n\n@app.middleware(\"http\")\nasync def add_process_time_header(request: Request, call_next):\n start_time = time.time()\n response = await call_next(request)\n process_time = time.time() - start_time\n response.headers[\"X-Process-Time\"] = str(process_time)\n return response\n \n```\n#### Real-World Use Cases of Middlewares:\n\n- Adding custom functionality to the request-response cycle without disrupting the core framework\n- Separating concerns and keeping API endpoints focused on their core tasks while handling shared operations through middleware components\n- Enabling the addition of authentication, error handling, data transformation, and more by extending the functionality of APIs in unique ways\n\nIn addition to built-in middleware, FastAPI allows the creation of custom middleware to extend its capabilities beyond the built-in options, offering a powerful way to enhance the functionality of APIs\n\n\n### Permissions in FastAPI\nIn FastAPI, permissions are used to control access to different parts of an application based on the user's role or other factors.\n\nHere's a simple example of using **permissions** in FastAPI to restrict access to a specific route based on the user's role:\n\n```\n\nfrom fastapi import FastAPI, Depends, HTTPException\nfrom fastapi.security import OAuth2PasswordBearer\nfrom typing import Optional\n\napp = FastAPI()\n\noauth2_scheme = OAuth2PasswordBearer(tokenUrl=\"token\")\n\nasync def get_current_user(token: str = Depends(oauth2_scheme)):\n # Implement logic to get user details from the token\n # Example: decode the token and fetch user details from the database\n # Return the user object if the token is valid, else raise an HTTPException\n return {\"username\": \"user1\", \"role\": \"admin\"}\n\ndef has_permission(user: dict = Depends(get_current_user), required_role: Optional[str] = None):\n if required_role and user[\"role\"] != required_role:\n raise HTTPException(status_code=403, detail=\"Permission denied\")\n return user\n\n@app.get(\"/admin\")\nasync def admin_route(current_user: dict = Depends(has_permission)):\n return {\"message\": \"Welcome admin!\"}\n\n```\n\n**Permissions** in FastAPI are crucial for implementing role-based access control (RBAC) and ensuring that users can only access the resources they are authorized to. Some real-world use cases include:\n\n- **Role-Based Access Control (RBAC):** Restricting access to certain routes or data based on the user's role, such as admin, user, or manager\n\n- **Row-Level Security:** Implementing fine-grained permissions based on the state or attributes of specific resources, such as allowing users to view, edit, or retract scientific papers based on the submission process state\n\n- **Custom Business Logic:** Enforcing custom business rules and access restrictions based on specific application requirements, such as allowing users to modify only their own items\n\nBy using permissions, FastAPI applications can ensure that sensitive data and functionality are protected, and that users are granted appropriate access based on their roles and other contextual factors.\n\n\n### Custom Validators in FastAPI\n\nCustom validators in FastAPI can be implemented using **Pydantic**, which is used for data validation in FastAPI. Pydantic allows you to define custom validation rules for request data. Here's a simple example of using custom validators in FastAPI with Pydantic:\n\n```\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel, Field, ValidationError\n\napp = FastAPI()\n\nclass Item(BaseModel):\n name: str\n price: float\n\n @property\n def price_must_be_positive(cls):\n if cls.price <= 0:\n raise ValueError('price must be positive')\n\n@app.post(\"/items/\")\nasync def create_item(item: Item):\n try:\n item.price_must_be_positive\n except ValueError as e:\n return {\"error\": str(e)}\n return {\"item_name\": item.name, \"item_price\": item.price}\n \n```\nIn this example, a custom validator **price_must_be_positive** is defined within the **Item** class to ensure that the price of an item is positive. If the validation fails, a **ValueError** is raised, and an appropriate error message is returned.\n\n#### Use Cases in Real-World Applications\n\nCustom validators in FastAPI can be used in various real-world scenarios, such as:\n\n- **Data Integrity Checks:** Ensuring that the incoming data meets specific business rules or constraints, such as validating the format of user input, ensuring the correctness of submitted data, or enforcing specific conditions on the input data\n- **Complex Data Validation:** Implementing custom validation logic for complex data structures or interdependent fields, such as validating relationships between different fields or performing custom checks based on multiple input parameters\n- **Integration with External Services:** Validating data against external services or APIs, such as verifying the correctness of input data before making requests to external systems or services\n\nBy using custom validators, FastAPI applications can enforce specific data validation rules tailored to their business requirements, ensuring the integrity and correctness of incoming data.\n\n\n### FastAPI BaseSettings\nIn FastAPI, **BaseSettings** is a class provided by **Pydantic** for managing application settings and environment variables. It allows you to define a settings model with default values and data types, and then load and use these settings throughout your application. Here's a simple example of using **BaseSettings** in FastAPI:\n\n```\nfrom fastapi import FastAPI\nfrom pydantic import BaseSettings\n\nclass Settings(BaseSettings):\n app_name: str = \"Awesome API\"\n admin_email: str\n items_per_user: int = 50\n\nsettings = Settings()\n\napp = FastAPI()\n\n@app.get(\"/info\")\nasync def info():\n return {\n \"app_name\": settings.app_name,\n \"admin_email\": settings.admin_email,\n \"items_per_user\": settings.items_per_user\n }\n```\nIn this example, the **Settings** class inherits from **BaseSettings** and defines the application settings, including default values and data types. These settings can then be used throughout the FastAPI application.\n\n#### The use of **BaseSettings** in FastAPI has several real-world applications, including:\n\n- **Centralized Settings Management:** Providing a centralized and structured way to manage application settings, including environment-specific configurations, database credentials, API keys, and other parameters\n- **Environment Variable Handling:** Facilitating the loading and management of environment variables, allowing for easy configuration of application behavior across different deployment environments such as development, testing, and production\n\nBy using **BaseSettings**, FastAPI applications can maintain a clear and consistent approach to managing settings and configurations, ensuring that the application behaves predictably across different environments and deployment scenarios.\n\n\n\n### Dependency Caching\nIn FastAPI, dependency caching refers to the ability to cache the result of a dependency function so that it is not re-evaluated for each request. This can be useful for improving performance and reducing redundant computations. Here's a simple example of using dependency caching in FastAPI:\n\n```\nfrom fastapi import FastAPI, Depends\nfrom functools import lru_cache\n\napp = FastAPI()\n\n@lru_cache\ndef expensive_operation():\n # Simulate an expensive operation\n return \"result\"\n\nasync def get_cached_result(result: str = Depends(expensive_operation)):\n return {\"cached_result\": result}\n\n@app.get(\"/cached\")\nasync def cached_endpoint(response: dict = Depends(get_cached_result)):\n return response\n```\n\nIn this example, the **@lru_cache** decorator is used to cache the result of the **expensive_operation** function. The **get_cached_result** function then depends on the cached result of **expensive_operation**, and the cached result is used in the **/cached** endpoint.\n\n#### Use Cases in Real-World Applications\nDependency caching in FastAPI has several real-world applications, including:\n\n- **Performance Optimization:** Caching the result of computationally expensive operations, such as database queries, external API calls, or complex calculations, to improve response times and reduce server load\n- **Rate Limiting:** Using caching to enforce rate limits on certain operations or endpoints by storing and checking the frequency of requests within a specific time frame\n- **External Service Integration:** Caching the results of requests to external services or APIs to reduce latency and minimize the impact of service outages or slowdowns\n\nBy leveraging dependency caching, FastAPI applications can efficiently manage the results of dependencies, reduce redundant computations, and enhance overall system performance.\n\n\n### Rate Limiting\n**Rate limiting** in FastAPI allows you to restrict the number of requests a client can make to an API within a given time frame. This can be achieved using third-party packages such as **fastapi-limiter**. Here's a simple example of using rate limiting in FastAPI with fastapi-limiter:\n\n```\nfrom fastapi import FastAPI\nfrom fastapi_limiter import FastAPILimiter\nfrom fastapi_limiter.depends import RateLimiter\n\napp = FastAPI()\n\n# Initialize the rate limiter\nFastAPILimiter.init()\n\n@app.get(\"/limited\")\n@RateLimiter(times=5, seconds=60)\nasync def limited_endpoint():\n return {\"message\": \"This endpoint is rate-limited\"}\n \n```\n\nIn this example, the **fastapi-limiter** package is used to apply rate limiting to the **/limited** endpoint, allowing a maximum of 5 requests per 60 seconds.\n\n#### Use Cases in Real-World Applications\n\nRate limiting in FastAPI has several real-world applications, including:\n- **Protection Against Abuse:** Preventing abuse and misuse of APIs by limiting the number of requests from a single client within a specific time period, safeguarding the API from excessive traffic and potential denial-of-service attacks\n- **Throttling:** Managing the flow of requests to ensure that the API server is not overwhelmed by a large number of requests, maintaining system stability and preventing performance degradation during peak usage periods\n\nBy implementing rate limiting, FastAPI applications can effectively manage and control the volume of incoming requests, ensuring the stability, security, and fair usage of the API.\n\n\n### Cache in FastAPI\nFastAPI provides various options for caching responses and function results, including third-party packages such as **fastapi-cache**. This tool allows you to cache FastAPI responses and function results, with support for backends like Redis, Memcached, and Amazon DynamoDB\nHere's a simple example of using **fastapi-cache** to cache a FastAPI response:\n\n```\nfrom fastapi import FastAPI\nfrom fastapi_cache import FastAPICache\nfrom fastapi_cache.backends.redis import RedisBackend\nfrom fastapi_cache.decorator import cache\nimport aioredis\n\napp = FastAPI()\n\n# Initialize the cache with Redis as the backend\n@app.on_event(\"startup\")\nasync def startup():\n redis = await aioredis.create_redis_pool('redis://localhost')\n FastAPICache.init(RedisBackend(redis), prefix=\"fastapi-cache\")\n\n# Cache the response of the endpoint\n@app.get(\"/\")\n@cache()\nasync def cached_endpoint():\n return {\"message\": \"This response is cached\"}\n\n```\n\n#### Use Cases in Real-World Applications\nCaching in FastAPI has several real-world applications, including:\n- **Performance Optimization:** Caching responses to reduce the computational load and improve the response time of frequently accessed endpoints, enhancing the overall performance of the API\n- **Database Query Results:** Caching the results of database queries to minimize the load on the database and improve the responsiveness of data retrieval operations\n- **External API Responses:** Caching responses from external APIs to reduce latency and minimize the impact of external service outages or slowdowns, ensuring a more reliable and responsive API\n\nBy leveraging caching in FastAPI, applications can efficiently manage and optimize the handling of responses and data, leading to improved performance and a better user experience.\n\n\n### Custom Exceptions in FastAPI\n\nCustom exceptions in FastAPI allow you to define and handle application-specific errors in a structured manner. You can create custom exception classes and handle them using FastAPI's exception handling mechanisms. Here's a simple example of using custom exceptions in FastAPI:\n\n```angular2html\nfrom fastapi import FastAPI, HTTPException\n\napp = FastAPI()\n\nclass CustomException(Exception):\n def __init__(self, detail: str):\n self.detail = detail\n\n@app.get(\"/custom_exception\")\nasync def custom_exception_endpoint():\n raise CustomException(detail=\"Custom exception message\")\n\n```\nIn this example, a custom exception class **CustomException** is defined, and it is raised within the **/custom_exception** endpoint. You can then handle this custom exception using FastAPI's exception handling mechanisms.\n\n#### Use Cases in Real-World Applications\nCustom exceptions in FastAPI have several real-world applications, including:\n\n- **Application-Specific Errors:** Defining custom exceptions to represent specific error conditions or business logic failures within the application, providing a structured way to handle and communicate these errors to clients\n\n- **Error Abstraction:** Abstracting and encapsulating complex error handling logic into custom exception classes, promoting code modularity and maintainability by centralizing error management\n\n- **Consistent Error Responses:** Standardizing error responses by using custom exceptions to represent different error scenarios, ensuring a consistent and predictable API behavior for clients\n\nBy utilizing custom exceptions, FastAPI applications can effectively manage and communicate application-specific errors, enhancing the robustness and reliability of the API.\n\n\n### Optimization techniques in FastAPI\nFastAPI is a high-performance web framework for building REST APIs in Python. It provides various optimization techniques to enhance the performance of applications.\n\nSome of the Optimization Techniques\n\n- **Asynchronous Programming:** Utilize asynchronous functions to handle heavy request payloads and I/O bound operations\n\n- **Use of Pydantic for Data Validation:** Leverage Pydantic models for data validation and conversion\n\n- **Dependency Caching:** Reuse dependencies and cache their results within a request's scope to avoid recalculating them\n\n- **BackgroundTasks:** Use BackgroundTasks for handling tasks that can be run in the background to improve response times\n\n- **Optimizing CPU Intensive Tasks:** Send CPU intensive tasks to workers in another process for optimization\n\n- **Caching Responses:** Implement response caching to store the results of expensive computations and serve them directly for subsequent identical requests\n\n- **Optimizing JSON Responses:** Utilize FastAPI's JSON response classes for efficient serialization and deserialization of JSON data\n\n- **Asynchronous Database Operations:** Use asynchronous database drivers and queries to optimize database interactions and improve overall request handling\n\n- **Project Structure:** Organize the project structure following best practices to enhance maintainability and performance\n\n- **Use of APIRouter:** Utilize APIRouter to modularize and organize route handling, especially for larger applications with multiple files\n\n- **Observability Tools:** Employ observability tools like New Relic to gain insights into performance issues and optimize FastAPI applications\n\n- **Custom Base Model:** Implement a custom base model from the beginning to ensure consistency and efficiency in data handling\n\n- **Use of Pydantic's BaseSettings for Configs:** Leverage Pydantic's BaseSettings for efficient management of application configurations\n\n- **Optimizing File Handling:** Save files in chunks to optimize file handling and improve overall performance\n\n- **Avoid Unnecessary Asynchronous Operations:** Do not make routes async if there are only blocking I/O operations, as unnecessary async operations can impact performance\n\n- **Careful Usage of Dynamic Pydantic Fields:** Exercise caution when using dynamic Pydantic fields to avoid potential performance impacts\n\n- **Dependency Calls Caching:** Cache dependency calls to improve performance by reusing previously calculated results\n\n- **Documentation:** Ensure comprehensive documentation to facilitate understanding and usage, contributing to efficient development and performance\n\n\n### Concurrency and Parallelism In FastAPI\nConcurrency and parallelism are essential concepts in FastAPI for handling multiple tasks simultaneously and improving performance. Concurrency refers to the ability to execute multiple tasks in overlapping time periods, while parallelism involves executing multiple tasks simultaneously. \nReal-world use cases of concurrency and parallelism in FastAPI include:\n\n- **Handling Multiple Requests:** Concurrency allows the server to handle multiple incoming requests simultaneously, improving the responsiveness of the application.\n- **Asynchronous I/O Operations:** Concurrency is beneficial for I/O-bound operations such as reading from databases, making API calls, or accessing files, as it allows the server to perform other tasks while waiting for I/O operations to complete.\n- **Parallel Processing:** For CPU-bound tasks like data processing or machine learning computations, parallelism can be used to execute multiple tasks concurrently, leveraging multi-core processors for improved performance.\n- **Real-time Data Streaming:** Concurrency enables the server to handle real-time data streaming, such as sending continuous updates to clients while simultaneously processing other tasks.\n- **Background Tasks:** Asynchronous operations are useful for executing background tasks, such as sending emails, processing notifications, or performing periodic maintenance tasks without blocking the main application flow\n\n### API Documentation Best Practices\n\nFastAPI automatically generates interactive API documentation, but following best practices enhances its usefulness:\n\n**Customizing OpenAPI Schema:**\n\n```python\nfrom fastapi import FastAPI\nfrom fastapi.openapi.utils import get_openapi\n\napp = FastAPI(\n title=\"My API\",\n description=\"A comprehensive API for managing resources\",\n version=\"1.0.0\",\n)\n\ndef custom_openapi():\n if app.openapi_schema:\n return app.openapi_schema\n openapi_schema = get_openapi(\n title=\"My API\",\n version=\"1.0.0\",\n description=\"Custom API documentation\",\n routes=app.routes,\n )\n app.openapi_schema = openapi_schema\n return app.openapi_schema\n\napp.openapi = custom_openapi\n```\n\n**Adding Examples and Descriptions:**\n\n```python\nfrom pydantic import BaseModel, Field\n\nclass Item(BaseModel):\n name: str = Field(..., description=\"Item name\", example=\"Widget\")\n price: float = Field(..., gt=0, description=\"Item price in USD\", example=29.99)\n description: str = Field(None, description=\"Item description\", example=\"A useful widget\")\n\n@app.post(\"/items/\", response_model=Item, summary=\"Create an item\", \n description=\"Create a new item in the system\",\n response_description=\"The created item\")\nasync def create_item(item: Item):\n \"\"\"Create a new item.\n \n - **name**: The name of the item\n - **price**: The price must be greater than 0\n - **description**: Optional description\n \"\"\"\n return item\n```\n\n**Tagging and Grouping:**\n\n```python\n@app.post(\"/users/\", tags=[\"users\"], summary=\"Create user\")\nasync def create_user(user: User):\n pass\n\n@app.get(\"/users/\", tags=[\"users\"], summary=\"List users\")\nasync def list_users():\n pass\n\n@app.post(\"/items/\", tags=[\"items\"], summary=\"Create item\")\nasync def create_item(item: Item):\n pass\n```\n\n**Use Cases:**\n\n1. **API Discovery:** Help developers understand available endpoints\n2. **Testing:** Interactive docs allow testing endpoints directly\n3. **Documentation:** Automatic documentation reduces maintenance\n4. **Client Generation:** OpenAPI schema can generate client libraries\n\n### Testing in FastAPI\n\nComprehensive testing ensures FastAPI applications work correctly:\n\n**Test Client:**\n\n```python\nfrom fastapi.testclient import TestClient\nfrom fastapi import FastAPI\n\napp = FastAPI()\n\n@app.get(\"/\")\ndef read_root():\n return {\"message\": \"Hello World\"}\n\n# Tests\ndef test_read_root():\n client = TestClient(app)\n response = client.get(\"/\")\n assert response.status_code == 200\n assert response.json() == {\"message\": \"Hello World\"}\n```\n\n**Testing with Dependencies:**\n\n```python\nfrom fastapi import Depends\nfrom unittest.mock import Mock\n\ndef get_db():\n # Mock database dependency\n return Mock()\n\n@app.get(\"/items/\")\ndef read_items(db=Depends(get_db)):\n return db.query_items()\n\ndef test_read_items():\n client = TestClient(app)\n response = client.get(\"/items/\")\n assert response.status_code == 200\n```\n\n**Async Testing:**\n\n```python\nimport pytest\nfrom httpx import AsyncClient\n\n@pytest.mark.asyncio\nasync def test_async_endpoint():\n async with AsyncClient(app=app, base_url=\"http://test\") as ac:\n response = await ac.get(\"/async-endpoint/\")\n assert response.status_code == 200\n```\n\n**Use Cases:**\n\n1. **Unit Testing:** Test individual endpoints and functions\n2. **Integration Testing:** Test complete request/response cycles\n3. **API Contract Testing:** Ensure API contracts are maintained\n4. **Performance Testing:** Test endpoint performance under load\n\n### Security Best Practices in FastAPI\n\nSecurity is critical for API applications:\n\n**Authentication with JWT:**\n\n```python\nfrom fastapi import Depends, HTTPException, status\nfrom fastapi.security import OAuth2PasswordBearer\nfrom jose import JWTError, jwt\n\noauth2_scheme = OAuth2PasswordBearer(tokenUrl=\"token\")\n\nSECRET_KEY = \"your-secret-key\"\nALGORITHM = \"HS256\"\n\nasync def get_current_user(token: str = Depends(oauth2_scheme)):\n credentials_exception = HTTPException(\n status_code=status.HTTP_401_UNAUTHORIZED,\n detail=\"Could not validate credentials\",\n headers={\"WWW-Authenticate\": \"Bearer\"},\n )\n try:\n payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])\n username: str = payload.get(\"sub\")\n if username is None:\n raise credentials_exception\n except JWTError:\n raise credentials_exception\n return username\n\n@app.get(\"/protected/\")\nasync def protected_route(current_user: str = Depends(get_current_user)):\n return {\"user\": current_user}\n```\n\n**Input Validation:**\n\n```python\nfrom pydantic import BaseModel, validator, EmailStr\n\nclass UserCreate(BaseModel):\n email: EmailStr\n password: str\n \n @validator('password')\n def validate_password(cls, v):\n if len(v) < 8:\n raise ValueError('Password must be at least 8 characters')\n return v\n```\n\n**Rate Limiting:**\n\n```python\nfrom slowapi import Limiter, _rate_limit_exceeded_handler\nfrom slowapi.util import get_remote_address\nfrom slowapi.errors import RateLimitExceeded\n\nlimiter = Limiter(key_func=get_remote_address)\napp.state.limiter = limiter\napp.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)\n\n@app.get(\"/api/\")\n@limiter.limit(\"5/minute\")\nasync def limited_endpoint(request: Request):\n return {\"message\": \"This is rate limited\"}\n```\n\n**CORS Configuration:**\n\n```python\nfrom fastapi.middleware.cors import CORSMiddleware\n\napp.add_middleware(\n CORSMiddleware,\n allow_origins=[\"https://example.com\"],\n allow_credentials=True,\n allow_methods=[\"*\"],\n allow_headers=[\"*\"],\n)\n```\n\n**Use Cases:**\n\n1. **Authentication:** Secure API access with tokens\n2. **Authorization:** Control access to resources\n3. **Input Sanitization:** Prevent injection attacks\n4. **Rate Limiting:** Prevent abuse and ensure fair usage\n\n### Logging and Monitoring in FastAPI\n\nEffective logging and monitoring help track API behavior:\n\n**Structured Logging:**\n\n```python\nimport logging\nimport sys\nfrom pythonjsonlogger import jsonlogger\n\n# Configure JSON logger\nlogHandler = logging.StreamHandler(sys.stdout)\nformatter = jsonlogger.JsonFormatter()\nlogHandler.setFormatter(formatter)\nrootLogger = logging.getLogger()\nrootLogger.addHandler(logHandler)\nrootLogger.setLevel(logging.INFO)\n\nlogger = logging.getLogger(__name__)\n\n@app.get(\"/items/\")\nasync def get_items():\n logger.info(\"Fetching items\", extra={\"endpoint\": \"/items/\"})\n return {\"items\": []}\n```\n\n**Request Logging Middleware:**\n\n```python\nimport time\nfrom fastapi import Request\n\n@app.middleware(\"http\")\nasync def log_requests(request: Request, call_next):\n start_time = time.time()\n response = await call_next(request)\n process_time = time.time() - start_time\n logger.info(\n \"Request processed\",\n extra={\n \"method\": request.method,\n \"url\": str(request.url),\n \"status_code\": response.status_code,\n \"process_time\": process_time,\n }\n )\n return response\n```\n\n**Prometheus Metrics:**\n\n```python\n# Install: pip install prometheus-fastapi-instrumentator\nfrom prometheus_fastapi_instrumentator import Instrumentator\n\nInstrumentator().instrument(app).expose(app)\n```\n\n**Use Cases:**\n\n1. **Error Tracking:** Monitor and debug production errors\n2. **Performance Monitoring:** Track response times and throughput\n3. **Audit Logging:** Log important API operations\n4. **Analytics:** Understand API usage patterns\n\n## General Topics:\n\n### REST API Design Principles\n\nFollowing REST principles ensures APIs are intuitive and maintainable:\n\n**Resource-Based URLs:**\n\n```python\n# Good\nGET /api/users/ # List users\nPOST /api/users/ # Create user\nGET /api/users/123/ # Get user\nPUT /api/users/123/ # Update user\nDELETE /api/users/123/ # Delete user\n\n# Bad\nGET /api/getUser/123\nPOST /api/createUser\n```\n\n**HTTP Status Codes:**\n\n```python\nfrom fastapi import status\n\n@app.post(\"/users/\", status_code=status.HTTP_201_CREATED)\nasync def create_user(user: User):\n return user\n\n@app.get(\"/users/\", status_code=status.HTTP_200_OK)\nasync def list_users():\n return []\n\n@app.get(\"/users/999/\", status_code=status.HTTP_404_NOT_FOUND)\nasync def get_user(user_id: int):\n raise HTTPException(status_code=404, detail=\"User not found\")\n```\n\n**Versioning:**\n\n```python\n# URL versioning\n@app.get(\"/v1/users/\")\nasync def list_users_v1():\n pass\n\n@app.get(\"/v2/users/\")\nasync def list_users_v2():\n pass\n\n# Header versioning\n@app.get(\"/users/\", headers={\"API-Version\": \"v1\"})\nasync def list_users():\n pass\n```\n\n**Pagination:**\n\n```python\nfrom fastapi import Query\n\n@app.get(\"/items/\")\nasync def list_items(\n skip: int = Query(0, ge=0),\n limit: int = Query(10, ge=1, le=100)\n):\n return {\"items\": [], \"skip\": skip, \"limit\": limit}\n```\n\n**Use Cases:**\n\n1. **API Consistency:** Standard patterns make APIs easier to use\n2. **Scalability:** RESTful design supports growth\n3. **Maintainability:** Clear structure simplifies updates\n4. **Developer Experience:** Intuitive APIs reduce learning curve\n\n### Deployment and DevOps\n\n#### Docker and Containerization\n\nContainerization ensures consistent deployment across environments:\n\n**Dockerfile for FastAPI:**\n\n```dockerfile\nFROM python:3.11-slim\n\nWORKDIR /app\n\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\nCOPY . .\n\nCMD [\"uvicorn\", \"main:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]\n```\n\n**Dockerfile for Django:**\n\n```dockerfile\nFROM python:3.11-slim\n\nWORKDIR /app\n\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\nCOPY . .\n\nRUN python manage.py collectstatic --noinput\n\nCMD [\"gunicorn\", \"myproject.wsgi:application\", \"--bind\", \"0.0.0.0:8000\"]\n```\n\n**Docker Compose:**\n\n```yaml\nversion: '3.8'\n\nservices:\n web:\n build: .\n ports:\n - \"8000:8000\"\n environment:\n - DATABASE_URL=postgresql://user:pass@db:5432/mydb\n depends_on:\n - db\n \n db:\n image: postgres:15\n environment:\n - POSTGRES_DB=mydb\n - POSTGRES_USER=user\n - POSTGRES_PASSWORD=pass\n volumes:\n - postgres_data:/var/lib/postgresql/data\n\nvolumes:\n postgres_data:\n```\n\n**Use Cases:**\n\n1. **Environment Consistency:** Same environment in dev, staging, production\n2. **Isolation:** Applications don't interfere with each other\n3. **Scalability:** Easy to scale containers horizontally\n4. **CI/CD Integration:** Containers work seamlessly with CI/CD pipelines\n\n#### CI/CD Pipelines\n\nAutomated testing and deployment improve development workflow:\n\n**GitHub Actions Example:**\n\n```yaml\n# .github/workflows/ci.yml\nname: CI\n\non: [push, pull_request]\n\njobs:\n test:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python\n uses: actions/setup-python@v4\n with:\n python-version: '3.11'\n - name: Install dependencies\n run: |\n pip install -r requirements.txt\n pip install pytest\n - name: Run tests\n run: pytest\n - name: Run linter\n run: |\n pip install black flake8\n black --check .\n flake8 .\n \n deploy:\n needs: test\n runs-on: ubuntu-latest\n if: github.ref == 'refs/heads/main'\n steps:\n - uses: actions/checkout@v3\n - name: Deploy to production\n run: |\n # Deployment commands\n```\n\n**GitLab CI Example:**\n\n```yaml\n# .gitlab-ci.yml\nstages:\n - test\n - deploy\n\ntest:\n stage: test\n image: python:3.11\n script:\n - pip install -r requirements.txt\n - pytest\n - black --check .\n - flake8 .\n\ndeploy:\n stage: deploy\n script:\n - echo \"Deploying to production\"\n only:\n - main\n```\n\n**Use Cases:**\n\n1. **Automated Testing:** Run tests on every commit\n2. **Code Quality:** Enforce code standards automatically\n3. **Deployment Automation:** Deploy to production automatically\n4. **Rollback Capability:** Easy to revert to previous versions\n\n#### Environment Management\n\nManaging different environments (dev, staging, production) effectively:\n\n**Environment Variables:**\n\n```python\n# .env\nDATABASE_URL=postgresql://user:pass@localhost/db\nSECRET_KEY=your-secret-key\nDEBUG=True\n\n# settings.py\nimport os\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nDATABASE_URL = os.getenv(\"DATABASE_URL\")\nSECRET_KEY = os.getenv(\"SECRET_KEY\")\nDEBUG = os.getenv(\"DEBUG\", \"False\") == \"True\"\n```\n\n**Configuration Management:**\n\n```python\n# config.py\nfrom pydantic_settings import BaseSettings\n\nclass Settings(BaseSettings):\n database_url: str\n secret_key: str\n debug: bool = False\n \n class Config:\n env_file = \".env\"\n env_file_encoding = \"utf-8\"\n\nsettings = Settings()\n```\n\n**Use Cases:**\n\n1. **Security:** Keep secrets out of code\n2. **Flexibility:** Different configs for different environments\n3. **Maintainability:** Centralized configuration management\n4. **Compliance:** Meet security and compliance requirements\n\n## Interview Preparation questions\n\nThis guide provides explanations, practical examples, questions, answers, and Python code snippets to solidify your understanding. Let's dive in!\n\n### PostgreSQL Querying\n\n- **Why Raw SQL?**\n\nWhile ORMs (like SQLAlchemy or Django ORM) are incredibly useful for rapid development and abstraction, understanding raw SQL is crucial for:\n\n* **Performance Tuning:** Writing highly optimized queries that an ORM might not generate.\n* **Complex Reporting:** Crafting intricate queries involving multiple joins, subqueries, window functions, or CTEs.\n* **Debugging:** Understanding the exact SQL being executed by an ORM or identifying database-level issues.\n* **Database Features:** Leveraging specific PostgreSQL features not directly exposed by all ORMs (e.g., specific index types, extensions).\n* **Legacy Systems:** Working with systems that may not use an ORM.\n\n\n- **Core SQL Concepts Review**\n\nBefore diving into questions, let's quickly recap essential SQL commands:\n\n* `SELECT`: Retrieves data from one or more tables.\n* `FROM`: Specifies the table(s) to query.\n* `WHERE`: Filters rows based on specified conditions.\n* `JOIN` (`INNER`, `LEFT`, `RIGHT`, `FULL OUTER`): Combines rows from two or more tables based on a related column.\n* `GROUP BY`: Groups rows that have the same values in specified columns into a summary row. Often used with aggregate functions (`COUNT`, `MAX`, `MIN`, `SUM`, `AVG`).\n* `HAVING`: Filters groups based on a specified condition (used *after* `GROUP BY`).\n* `ORDER BY`: Sorts the result set based on specified columns (`ASC`, `DESC`).\n* `LIMIT`: Restricts the number of rows returned.\n* `OFFSET`: Skips a specified number of rows before starting to return rows (often used with `LIMIT` for pagination).\n* `INSERT INTO`: Adds new rows to a table.\n* `UPDATE`: Modifies existing rows in a table.\n* `DELETE FROM`: Removes rows from a table.\n* **Transactions:** (`BEGIN`, `COMMIT`, `ROLLBACK`) Ensuring atomicity of operations.\n* **Indexes:** Data structures that improve the speed of data retrieval operations. Common types: B-tree (default), Hash, GiST, GIN.\n* **Subqueries:** Queries nested inside another query.\n* **Common Table Expressions (CTEs):** (`WITH ... AS ...`) Temporary, named result sets you can reference within a single statement. Improves readability for complex queries.\n* **Window Functions:** Perform calculations across a set of table rows that are somehow related to the current row (e.g., ranking, running totals).\n\n- **Example Schema**\n\nWe'll use the following simple schema for our practice questions:\n\n```sql\n-- Departments Table\nCREATE TABLE departments (\n id SERIAL PRIMARY KEY,\n name VARCHAR(100) NOT NULL UNIQUE,\n location VARCHAR(100)\n);\n\n-- Employees Table\nCREATE TABLE employees (\n id SERIAL PRIMARY KEY,\n name VARCHAR(100) NOT NULL,\n email VARCHAR(100) UNIQUE,\n salary DECIMAL(10, 2) CHECK (salary > 0),\n hire_date DATE DEFAULT CURRENT_DATE,\n department_id INTEGER REFERENCES departments(id) ON DELETE SET NULL -- Foreign key\n);\n\n-- Projects Table\nCREATE TABLE projects (\n id SERIAL PRIMARY KEY,\n name VARCHAR(150) NOT NULL UNIQUE,\n start_date DATE,\n deadline DATE\n);\n\n-- Employee-Project Junction Table (Many-to-Many)\nCREATE TABLE employee_projects (\n employee_id INTEGER REFERENCES employees(id) ON DELETE CASCADE,\n project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE,\n role VARCHAR(50),\n PRIMARY KEY (employee_id, project_id) -- Composite primary key\n);\n\n-- Sample Data (Conceptual)\nINSERT INTO departments (name, location) VALUES\n('Engineering', 'Building A'), ('Sales', 'Building B'), ('HR', 'Building A'), ('Marketing', 'Building B');\n\nINSERT INTO employees (name, email, salary, department_id) VALUES\n('Alice', 'alice@company.com', 90000, 1),\n('Bob', 'bob@company.com', 85000, 1),\n('Charlie', 'charlie@company.com', 70000, 2),\n('David', 'david@company.com', 72000, 2),\n('Eve', 'eve@company.com', 110000, 1),\n('Frank', 'frank@company.com', 60000, 3),\n('Grace', 'grace@company.com', 95000, NULL); -- No department assigned yet\n\nINSERT INTO projects (name, start_date, deadline) VALUES\n('Project Alpha', '2023-01-15', '2023-12-31'),\n('Project Beta', '2023-03-01', '2024-06-30'),\n('Project Gamma', '2023-07-01', NULL);\n\nINSERT INTO employee_projects (employee_id, project_id, role) VALUES\n(1, 1, 'Developer'), (1, 2, 'Lead Developer'),\n(2, 1, 'Developer'),\n(3, 2, 'Sales Lead'),\n(5, 2, 'Tech Lead'),\n(5, 3, 'Architect');\n```\n\n- **Practice Questions & Answers (SQL)**\n\n**Q1: Basic Retrieval**\n\n* **Question:** Find the names and salaries of all employees earning more than $80,000.\n* **Answer (SQL):**\n ```sql\n SELECT name, salary\n FROM employees\n WHERE salary > 80000;\n ```\n \n**Q2: `ORDER BY` and `LIMIT`**\n\n* **Question:** Find the names and hire dates of the 3 most recently hired employees.\n* **Answer (SQL):**\n ```sql\n SELECT name, hire_date\n FROM employees\n ORDER BY hire_date DESC\n LIMIT 3;\n ```\n\n**Q3: `JOIN`**\n\n* **Question:** List the names of all employees and their corresponding department names. Only include employees who are assigned to a department.\n* **Answer (SQL):**\n ```sql\n SELECT e.name AS employee_name, d.name AS department_name\n FROM employees e\n INNER JOIN departments d ON e.department_id = d.id;\n ```\n\n\n\n**Q4: `LEFT JOIN`**\n\n* **Question:** List all employee names and their department names. Include employees even if they are not currently assigned to a department.\n* **Answer (SQL):**\n ```sql\n SELECT e.name AS employee_name, d.name AS department_name\n FROM employees e\n LEFT JOIN departments d ON e.department_id = d.id;\n ```\n* **Explanation:** `LEFT JOIN` ensures all rows from the *left* table (`employees`) are included. If there's no match in the *right* table (`departments`), the columns from the right table will be `NULL`.\n\n\n\n**Q5: `GROUP BY` and Aggregate Functions**\n\n* **Question:** Find the number of employees in each department. Display the department name and the count.\n* **Answer (SQL):**\n ```sql\n SELECT d.name AS department_name, COUNT(e.id) AS employee_count\n FROM departments d\n LEFT JOIN employees e ON d.id = e.department_id\n GROUP BY d.name\n ORDER BY employee_count DESC;\n ```\n* **Note:** Using `LEFT JOIN` here ensures departments with zero employees are also listed (with a count of 0). Using `COUNT(e.id)` correctly counts employees, ignoring `NULL`s from the `LEFT JOIN`.\n\n\n**Q6: `GROUP BY` and `HAVING`**\n\n* **Question:** Find the departments with an average employee salary greater than $85,000. Display the department name and the average salary.\n* **Answer (SQL):**\n ```sql\n SELECT d.name AS department_name, AVG(e.salary) AS average_salary\n FROM employees e\n JOIN departments d ON e.department_id = d.id\n GROUP BY d.name\n HAVING AVG(e.salary) > 85000;\n ```\n* **Explanation:** `WHERE` filters rows *before* aggregation, while `HAVING` filters groups *after* aggregation.\n\n\n**Q7: Subquery in `WHERE` Clause**\n\n* **Question:** Find the names of employees who earn more than the overall average salary of all employees.\n* **Answer (SQL):**\n ```sql\n SELECT name, salary\n FROM employees\n WHERE salary > (SELECT AVG(salary) FROM employees);\n ```\n\n**Q8: Subquery in `SELECT` Clause (Correlated Subquery)**\n\n* **Question:** For each employee, show their name, salary, and the average salary of their department.\n* **Answer (SQL):**\n ```sql\n SELECT\n e.name AS employee_name,\n e.salary,\n (SELECT AVG(salary) FROM employees e2 WHERE e2.department_id = e.department_id) AS department_avg_salary\n FROM employees e;\n ```\n* **Note:** Correlated subqueries can sometimes be less performant than joins or window functions for this type of task, especially on large tables.\n\n\n**Q9: Many-to-Many Join**\n\n* **Question:** List all employees working on 'Project Beta'. Show employee name and their role on the project.\n* **Answer (SQL):**\n ```sql\n SELECT e.name AS employee_name, ep.role\n FROM employees e\n JOIN employee_projects ep ON e.id = ep.employee_id\n JOIN projects p ON ep.project_id = p.id\n WHERE p.name = 'Project Beta';\n ```\n**Q10: Common Table Expression (CTE)**\n\n* **Question:** Find departments where the maximum salary is greater than $100,000. Use a CTE to first find the maximum salary per department.\n* **Answer (SQL):**\n ```sql\n WITH DepartmentMaxSalary AS (\n SELECT\n department_id,\n MAX(salary) AS max_salary\n FROM employees\n WHERE department_id IS NOT NULL\n GROUP BY department_id\n )\n SELECT d.name AS department_name, dms.max_salary\n FROM departments d\n JOIN DepartmentMaxSalary dms ON d.id = dms.department_id\n WHERE dms.max_salary > 100000;\n ```\n\n**Q11: Window Function (`RANK`)**\n\n* **Question:** Rank employees within each department based on their salary (highest salary gets rank 1). Display department name, employee name, salary, and rank.\n* **Answer (SQL):**\n ```sql\n SELECT\n d.name AS department_name,\n e.name AS employee_name,\n e.salary,\n RANK() OVER (PARTITION BY e.department_id ORDER BY e.salary DESC) AS salary_rank\n FROM employees e\n JOIN departments d ON e.department_id = d.id\n ORDER BY d.name, salary_rank;\n ```\n* **Explanation:** `PARTITION BY e.department_id` divides the rows into partitions (one for each department). `ORDER BY e.salary DESC` sorts rows within each partition. `RANK()` assigns the rank based on this order. `DENSE_RANK()` is similar but doesn't leave gaps for ties. `ROW_NUMBER()` assigns unique numbers regardless of ties.\n\n**Q12: Data Modification (`INSERT`, `UPDATE`, `DELETE`)**\n\n* **Question:**\n 1. Add a new department 'Finance' in 'Building C'.\n 2. Update 'Charlie's salary to $75,000.\n 3. Remove the employee named 'Frank'.\n* **Answer (SQL):**\n ```sql\n -- 1. Insert new department\n INSERT INTO departments (name, location)\n VALUES ('Finance', 'Building C');\n\n -- 2. Update Charlie's salary (assuming email is unique identifier if name isn't)\n UPDATE employees\n SET salary = 75000\n WHERE email = 'charlie@company.com'; -- Use a unique identifier like id or email\n\n -- 3. Delete Frank (use a unique identifier)\n DELETE FROM employees\n WHERE name = 'Frank' AND email = 'frank@company.com'; -- Be specific\n ```\n* **Caution:** `DELETE` operations are permanent. Always be careful and use specific `WHERE` clauses, preferably targeting primary keys or unique constraints. Transactions (`BEGIN`, `COMMIT`, `ROLLBACK`) are essential for safety.\n\n**Q13: Indexing Concept**\n\n* **Question:** When would you add an index to the `employees` table? Explain your reasoning for choosing a specific column(s).\n* **Answer (Conceptual):**\n * You would add an index to columns frequently used in `WHERE` clauses, `JOIN` conditions, or `ORDER BY` clauses to speed up data retrieval.\n * **`id` (Primary Key):** PostgreSQL automatically creates a unique B-tree index on primary keys. This is essential for fast lookups and enforcing uniqueness.\n * **`department_id` (Foreign Key):** Definitely index this column. It's used in `JOIN` operations with the `departments` table and potentially in `WHERE` clauses (e.g., `WHERE department_id = X`). A B-tree index is suitable here.\n * **`email` (Unique Constraint):** PostgreSQL often automatically creates an index for unique constraints (typically B-tree) to efficiently check for duplicates during inserts/updates and speed up lookups based on email.\n * **`name`:** If you frequently search or sort employees by name (`WHERE name = '...'` or `ORDER BY name`), an index on this column could be beneficial. A B-tree index works well for range queries and exact matches.\n * **`salary`:** If you often filter or sort by salary (`WHERE salary > X`, `ORDER BY salary`), indexing this column can improve performance. A B-tree index is appropriate.\n * **`hire_date`:** Similar to `salary`, if filtering or sorting by `hire_date` is common, an index helps.\n * **Composite Index:** If you frequently filter by `department_id` AND `salary` together (e.g., `WHERE department_id = X AND salary > Y`), a composite index on `(department_id, salary)` might be more efficient than two separate indexes. The order matters.\n * **Trade-offs:** Indexes speed up reads (`SELECT`) but slow down writes (`INSERT`, `UPDATE`, `DELETE`) because the index also needs to be updated. They also consume disk space. Only index columns where the performance benefit outweighs the cost.\n\n\n**Q14: Transaction Concept**\n\n* **Question:** Imagine you need to transfer an employee ('Alice') from 'Engineering' to 'Sales' and simultaneously update her role on 'Project Alpha' to 'Consultant'. Why is it important to use a transaction for these operations?\n* **Answer (Conceptual):**\n * A transaction groups multiple SQL statements into a single, atomic unit of work. It guarantees **ACID** properties (Atomicity, Consistency, Isolation, Durability).\n * **Atomicity:** In this scenario, both updating the employee's `department_id` and updating their role in `employee_projects` must succeed together, or neither should happen. If the first update succeeds but the second fails (e.g., due to a constraint violation or connection drop), a transaction ensures the first update is rolled back, leaving the database in its original consistent state. Without a transaction, you could end up with inconsistent data (Alice in Sales but still listed as 'Developer' on the project, or vice-versa depending on the order and failure point).\n * **Consistency:** The transaction ensures the database transitions from one valid state to another, respecting all constraints.\n * **Isolation:** If other users are querying the data concurrently, a transaction ensures they see the data either *before* the changes or *after* both changes are committed, preventing them from seeing intermediate, inconsistent states (e.g., Alice listed in Sales but still having her old Engineering role on the project).\n * **Durability:** Once a transaction is committed, the changes are permanent and will survive system crashes.\n * **Syntax Example:**\n ```sql\n BEGIN; -- Start transaction\n\n UPDATE employees\n SET department_id = (SELECT id FROM departments WHERE name = 'Sales')\n WHERE email = 'alice@company.com';\n\n UPDATE employee_projects\n SET role = 'Consultant'\n WHERE employee_id = (SELECT id FROM employees WHERE email = 'alice@company.com')\n AND project_id = (SELECT id FROM projects WHERE name = 'Project Alpha');\n\n -- Add more related operations if needed...\n\n COMMIT; -- Apply changes permanently if all succeed\n -- Or ROLLBACK; if an error occurs or check fails\n ```\n\n\n\n\n\n\n\n\n\n### Algorithmic Problem Solving\n\nSenior Python roles often require solving algorithmic problems efficiently. This involves understanding data structures, common algorithms, and complexity analysis (Big O notation).\n\n#### Common Data Structures Review\n\nBe comfortable using and knowing the trade-offs of:\n\n* **Lists:** Ordered, mutable sequences. O(1) append, O(n) insertion/deletion/search.\n* **Tuples:** Ordered, immutable sequences. Used for fixed collections.\n* **Dictionaries (Hash Maps):** Key-value pairs. Average O(1) insertion/deletion/lookup. Crucial for many algorithms.\n* **Sets:** Unordered collections of unique elements. Average O(1) add/remove/contains check. Useful for uniqueness and membership testing.\n* **Strings:** Immutable sequences of characters.\n* **(Conceptual) Stacks (LIFO):** Use `list.append()` for push, `list.pop()` for pop.\n* **(Conceptual) Queues (FIFO):** Use `collections.deque` for efficient O(1) appends and pops from both ends.\n* **(Conceptual) Heaps (Priority Queues):** Use `heapq` module. O(log n) push/pop smallest element.\n* **(Conceptual) Trees (Binary Search Trees, Tries):** Know their properties and traversal methods (in-order, pre-order, post-order, level-order/BFS).\n* **(Conceptual) Graphs:** Know representations (adjacency list, adjacency matrix) and traversal algorithms (BFS, DFS).\n\n#### Problem-Solving Strategy\n\n1. **Understand:** Clarify the problem, inputs, outputs, constraints, and edge cases. Ask questions!\n2. **Plan:** Think about different approaches. Consider data structures. Whiteboard or sketch ideas. Analyze potential time/space complexity.\n3. **Code:** Write clean, readable Python code. Use meaningful variable names.\n4. **Test:** Test with examples, edge cases (empty input, large input, specific values), and constraints.\n5. **Optimize:** If necessary, revisit your plan to improve time or space complexity.\n\n---\n\n#### Practice Problems & Solutions (Python)\n\n**Problem 1: Two Sum**\n\n**Statement:** Given an array of integers `nums` and an integer `target`, return *indices* of the two numbers such that they add up to `target`. Assume that each input would have *exactly one solution*, and you may not use the same element twice.\n\n**Example:** `nums = [2, 7, 11, 15]`, `target = 9` -> Output: `[0, 1]` (because `nums[0] + nums[1] == 9`)\n\n**Approach:** Iterate through the array. For each number `n`, check if `target - n` exists in a hash map (dictionary) that stores numbers encountered so far and their indices. If it exists, we found the pair. If not, add the current number `n` and its index to the map.\n\n**Python Code:**\n\n ```python\n def two_sum(nums: list[int], target: int) -> list[int]:\n \"\"\"Finds indices of two numbers that sum to target.\"\"\"\n num_map = {} # Stores {number: index}\n for index, num in enumerate(nums):\n complement = target - num\n if complement in num_map:\n return [num_map[complement], index]\n num_map[num] = index\n return [] # Should not be reached based on problem statement\n ```\n**Complexity:**\n\n* Time: O(n) - We iterate through the list once. Dictionary lookups/insertions are O(1) on average.\n* Space: O(n) - In the worst case, the dictionary stores all numbers.\n\n\n**Problem 2: Valid Parentheses**\n\n**Statement:** Given a string `s` containing just the characters `(`, `)`, `{`, `}`, `[` and `]`, determine if the input string is valid. An input string is valid if:\n 1. Open brackets must be closed by the same type of brackets.\n 2. Open brackets must be closed in the correct order.\n 3. Every close bracket has a corresponding open bracket of the same type.\n\n**Example:** `s = \"()[]{}\"` -> `True`; `s = \"(]\"` -> `False`; `s = \"{[]}\"` -> `True`\n\n**Approach:** Use a stack. Iterate through the string. If an opening bracket (`(`, `{`, `[`) is encountered, push it onto the stack. If a closing bracket (`)`, `}`, `]`) is encountered, check if the stack is empty or if the top of the stack is the corresponding opening bracket. If not, return `False`. If it matches, pop the stack. After iterating through the string, the stack should be empty for the string to be valid.\n\n**Python Code:**\n\n```python\n def is_valid_parentheses(s: str) -> bool:\n \"\"\"Checks if a string of parentheses is valid.\"\"\"\n stack = []\n mapping = {\")\": \"(\", \"}\": \"{\", \"]\": \"[\"}\n\n for char in s:\n if char in mapping: # It's a closing bracket\n # Pop from stack if not empty, otherwise use a dummy value\n top_element = stack.pop() if stack else '#'\n if mapping[char] != top_element:\n return False\n else: # It's an opening bracket\n stack.append(char)\n\n return not stack # Stack should be empty at the end\n```\n \n* **Complexity:**\n * Time: O(n) - We iterate through the string once. Stack operations are O(1).\n * Space: O(n) - In the worst case (e.g., `((((...))))`), the stack stores all opening brackets.\n\n\n\n**Problem 3: Longest Substring Without Repeating Characters**\n\n* **Statement:** Given a string `s`, find the length of the longest substring without repeating characters.\n* **Example:** `s = \"abcabcbb\"` -> `3` (\"abc\"); `s = \"bbbbb\"` -> `1` (\"b\"); `s = \"pwwkew\"` -> `3` (\"wke\")\n* **Approach:** Use a sliding window technique. Maintain a window `[left, right]` and a set (or dictionary) to keep track of characters currently in the window. Expand the window by moving `right`. If `s[right]` is already in the set, shrink the window from the left by moving `left` and removing `s[left]` from the set until `s[right]` is no longer a duplicate. Add `s[right]` to the set. Update the maximum length found so far (`right - left + 1`).\n* **Python Code:**\n ```python\n def length_of_longest_substring(s: str) -> int:\n \"\"\"Finds the length of the longest substring without repeating chars.\"\"\"\n char_set = set()\n left = 0\n max_length = 0\n\n for right in range(len(s)):\n # If char already in window, shrink from left until it's removed\n while s[right] in char_set:\n char_set.remove(s[left])\n left += 1\n # Add the new character to the set and update max length\n char_set.add(s[right])\n max_length = max(max_length, right - left + 1)\n\n return max_length\n ```\n* **Complexity:**\n * Time: O(n) - Although there's a nested `while` loop, each character is added and removed from the set at most once. Both `left` and `right` pointers traverse the string linearly.\n * Space: O(min(m, n)) - Where `n` is the length of the string and `m` is the size of the character set (e.g., 26 for lowercase English letters, or up to `n`). The space is used by the `char_set`.\n\n\n**Problem 4: Kth Largest Element in an Array**\n\n* **Statement:** Given an integer array `nums` and an integer `k`, return the `k`th largest element in the array. Note that it is the `k`th largest element in the sorted order, not the `k`th distinct element.\n* **Example:** `nums = [3, 2, 1, 5, 6, 4]`, `k = 2` -> `5`; `nums = [3, 2, 3, 1, 2, 4, 5, 5, 6]`, `k = 4` -> `4`\n* **Approach (Heap):** Use a min-heap. Iterate through the numbers. Push each number onto the heap. If the heap size exceeds `k`, pop the smallest element (the root of the min-heap). After iterating through all numbers, the root of the heap will be the `k`th largest element.\n* **Python Code:**\n ```python\n import heapq\n\n def find_kth_largest(nums: list[int], k: int) -> int:\n \"\"\"Finds the Kth largest element using a min-heap.\"\"\"\n # Create a min-heap\n min_heap = []\n for num in nums:\n heapq.heappush(min_heap, num)\n # If heap size exceeds k, remove the smallest element\n if len(min_heap) > k:\n heapq.heappop(min_heap)\n # The root of the heap is the Kth largest element\n return min_heap[0]\n\n # Alternative concise version using heapify:\n # def find_kth_largest_concise(nums: list[int], k: int) -> int:\n # return heapq.nlargest(k, nums)[-1]\n ```\n* **Complexity:**\n * Time: O(n log k) - We process `n` elements. Heap push/pop operations take O(log k) time since the heap size is capped at `k`. (`heapq.nlargest` is often O(n log k) as well). A Quickselect approach offers average O(n) but worst-case O(n^2).\n * Space: O(k) - To store the elements in the heap.\n\n\n**Problem 5: Coin Change**\n\n* **Statement:** You are given an integer array `coins` representing coins of different denominations and an integer `amount` representing a total amount of money. Return the *fewest* number of coins that you need to make up that amount. If that amount of money cannot be made up by any combination of the coins, return `-1`. Assume you have an infinite number of each kind of coin.\n* **Example:** `coins = [1, 2, 5]`, `amount = 11` -> `3` (11 = 5 + 5 + 1); `coins = [2]`, `amount = 3` -> `-1`\n* **Approach (Dynamic Programming):** Create a DP array `dp` of size `amount + 1`, initialized with a value indicating infinity (e.g., `amount + 1`) except `dp[0] = 0` (0 coins needed for amount 0). Iterate from `1` to `amount`. For each amount `i`, iterate through the `coins`. If a coin `c` is less than or equal to `i`, check if using this coin can lead to a smaller number of coins needed: `dp[i] = min(dp[i], 1 + dp[i - c])`. After the loops, if `dp[amount]` is still the infinity value, it means the amount is unreachable; otherwise, `dp[amount]` holds the minimum number of coins.\n* **Python Code:**\n ```python\n def coin_change(coins: list[int], amount: int) -> int:\n \"\"\"Finds the minimum number of coins to make a given amount (DP).\"\"\"\n # dp[i] will store the minimum coins needed for amount i\n # Initialize with a value larger than any possible result (amount + 1)\n dp = [amount + 1] * (amount + 1)\n dp[0] = 0 # Base case: 0 coins for amount 0\n\n for i in range(1, amount + 1):\n for coin in coins:\n if coin <= i:\n # If we use 'coin', we need 1 + dp[i - coin] coins\n dp[i] = min(dp[i], 1 + dp[i - coin])\n\n # If dp[amount] is still amount + 1, it means amount is unreachable\n return dp[amount] if dp[amount] <= amount else -1\n ```\n\n* **Complexity:**\n * Time: O(amount * num_coins) - We have nested loops iterating up to `amount` and through the `coins`.\n * Space: O(amount) - We use a DP array of size `amount + 1`.\n\n \nHere is another solution:\n\n* **Python Code:**\n ```python\n def coin_change(coins, amount):\n map_dict = {}\n fewest = 1000\n for element in coins:\n remainder = amount % element\n qu = amount // element\n if remainder == 0 or remainder in coins:\n fewest = min(qu + remainder, fewest)\n \n return -1 if fewest == 1000 else fewest\n ```\n\n\n**Problem 6: Merge Intervals**\n\n**Statement:** Given an array of intervals where intervals`[i] = [start_i, end_i]`, merge all overlapping intervals, and return an array of the non-overlapping intervals that cover all the intervals in the input.\n\n**Example:** `intervals = [[1,3],[2,6],[8,10],[15,18]] -> [[1,6],[8,10],[15,18]]` (because `[1,3]` and [2,6] overlap and merge into `[1,6]`). `intervals = [[1,4],[4,5]] -> [[1,5]]` (overlap at the boundary).\n\n**Approach:** First, sort the intervals based on their start times. Initialize a list merged_intervals with the first interval. Iterate through the sorted intervals starting from the second one. If the current interval overlaps with the last interval in merged_intervals (i.e., current_interval.start <= last_merged_interval.end), merge them by updating the end time of the last interval in merged_intervals to the maximum of the two end times. Otherwise, if they don't overlap, add the current interval to merged_intervals.\n\n**Python Code:**\n\n```python\ndef merge_intervals(intervals: list[list[int]]) -> list[list[int]]:\n if not intervals:\n return []\n\n # Sort intervals based on the start time\n intervals.sort(key=lambda x: x[0])\n\n merged_intervals = [intervals[0]]\n\n for i in range(1, len(intervals)):\n current_start, current_end = intervals[i]\n last_merged_start, last_merged_end = merged_intervals[-1]\n\n # Check for overlap\n if current_start <= last_merged_end:\n # Merge: update the end of the last merged interval\n merged_intervals[-1][1] = max(last_merged_end, current_end)\n else:\n # No overlap, add the current interval\n merged_intervals.append([current_start, current_end])\n\n return merged_intervals\n```\n\nComplexity:\n\nTime: O(n log n) - Dominated by the sorting step. The merging process is O(n).\n\nSpace: O(n) or O(log n) - O(n) for the merged_intervals list. The space complexity of sorting depends on the implementation (Timsort in Python is O(n), but sometimes considered O(log n) for recursion stack if not in-place).\n\n\n\n\n\n**Problem 7: Number of Islands**\n\n**Statement:** Given an **m x n** 2D binary grid grid which represents a map of '1's (land) and '0's (water), return the number of islands. An island is surrounded by water and is formed by connecting adjacent lands horizontally or vertically. You may assume all four edges of the grid are surrounded by water.\n\n**Example:**\n\n`\ngrid = [\n [\"1\",\"1\",\"1\",\"1\",\"0\"],\n [\"1\",\"1\",\"0\",\"1\",\"0\"],\n [\"1\",\"1\",\"0\",\"0\",\"0\"],\n [\"0\",\"0\",\"0\",\"0\",\"0\"]\n] -> 1\n`\n\n`\ngrid = [\n [\"1\",\"1\",\"0\",\"0\",\"0\"],\n [\"1\",\"1\",\"0\",\"0\",\"0\"],\n [\"0\",\"0\",\"1\",\"0\",\"0\"],\n [\"0\",\"0\",\"0\",\"1\",\"1\"]\n] -> 3\n`\n\n**Approach (DFS or BFS):** Iterate through each cell of the grid. If a cell contains '1' (land) and hasn't been visited yet (or is part of a found island), increment the island count. Then, start a traversal (DFS or BFS) from this cell to find all connected land cells belonging to this island. Mark these visited cells (e.g., change '1' to '0' or use a separate visited set) to avoid recounting them. Continue iterating through the grid.\n\n**Python Code (DFS):**\n\n```python\ndef num_islands(grid: list[list[str]]) -> int:\n \"\"\"Counts the number of islands in a grid using DFS.\"\"\"\n if not grid or not grid[0]:\n return 0\n\n rows, cols = len(grid), len(grid[0])\n num_islands = 0\n\n def dfs(r, c):\n # Check boundaries and if it's water or already visited (marked as '0')\n if r < 0 or c < 0 or r >= rows or c >= cols or grid[r][c] == '0':\n return\n # Mark the current cell as visited (by changing it to water)\n grid[r][c] = '0' \n # Explore neighbors\n dfs(r + 1, c)\n dfs(r - 1, c)\n dfs(r, c + 1)\n dfs(r, c - 1)\n\n for r in range(rows):\n for c in range(cols):\n if grid[r][c] == '1':\n num_islands += 1\n dfs(r, c) # Explore and mark the entire island\n\n return num_islands\n```\n\n\n**Complexity:**\n\nTime: `O(m * n)` - Each cell is visited at most a constant number of times.\n\nSpace: `O(m * n)` - In the worst case (grid full of land), the recursion depth for DFS could go up to mn. If using BFS, the queue could also store up to `O(mn)` cells. Also `O(m*n)` if using a separate visited set instead of modifying the grid.\n\n\n**Problem 8: Maximum Subarray**\n\n**Statement:** Given an integer array nums, find the contiguous subarray (containing at least one number) which has the largest sum and return its sum.\n\n**Example:** `nums = [-2,1,-3,4,-1,2,1,-5,4] -> 6` (subarray `[4,-1,2,1]`). `nums = [1] -> 1`. `nums = [5,4,-1,7,8] -> 23` (subarray `[5,4,-1,7,8]`).\n\n**Approach (Kadane's Algorithm):** Iterate through the array, keeping track of the maximum sum ending at the current position (current_max) and the overall maximum sum found so far (global_max). For each number, update current_max to be the maximum of the number itself OR the number plus the current_max from the previous position. This effectively decides whether to extend the previous subarray or start a new one. Update global_max whenever current_max is greater.\n\n**Python Code:**\n\n```python\ndef max_subarray(nums: list[int]) -> int:\n \"\"\"Finds the maximum sum of a contiguous subarray using Kadane's Algo.\"\"\"\n if not nums:\n return 0 # Or raise error, depends on constraints\n\n current_max = nums[0]\n global_max = nums[0]\n\n for i in range(1, len(nums)):\n # Decide whether to extend the current subarray or start a new one\n current_max = max(nums[i], current_max + nums[i])\n # Update the overall maximum sum found so far\n global_max = max(global_max, current_max)\n \n return global_max\n\n```\n\n\n**Complexity:**\n\nTime: O(n) - Single pass through the array.\n\nSpace: O(1) - Only a few variables are used.\n\n\n\n\n\n\n\n## Interview Questions\n\n### Problem 1: Database Transactions (Django & SQLAlchemy)\n\n**Statement:** Explain the concept of database transactions, why they are important, and demonstrate how to use them when performing multiple operations involving related models in both Django and SQLAlchemy.\n\n**Concept:**\n\nA database transaction is a sequence of one or more database operations treated as a single logical unit of work. This unit is governed by the ACID properties, primarily **Atomicity**. Atomicity ensures that either all operations within the transaction are successfully completed and the changes are committed to the database, or if any operation fails, the entire transaction is aborted, and all changes are rolled back to their state before the transaction began. This prevents partial updates and maintains data consistency, especially when dealing with interrelated data or multiple steps that must collectively succeed or fail.\n\n1. You need to perform multiple write operations that depend on each other (e.g., debiting one account and crediting another – both must happen).\n2. An operation involves modifying data based on the current state, and you want to prevent other processes from changing that state in between.\n3. You want a set of operations to be retryable or easily undone if something goes wrong.\n\n**Example Models (Author and Book):**\n\nWe'll use simple `Author` and `Book` models where a `Book` has a foreign key relationship to an `Author`. Creating an author and several books for them is a common scenario where a transaction is useful – you want either the author *and* all specified books to be created, or none of them if any step fails (e.g., a validation error on a book title).\n\n**Django ORM:**\n\nDjango manages transactions automatically for individual `save()` or `create()` calls (usually implicit), but for a block of multiple operations, you typically use `django.db.transaction.atomic()`. This context manager guarantees atomicity for the operations performed within the `with` block.\n\n```python\n\n# Assume these models are defined in your Django app's models.py\n# from django.db import models\n\n# class Author(models.Model):\n# name = models.CharField(max_length=100)\n\n# class Book(models.Model):\n# title = models.CharField(max_length=200)\n# author = models.ForeignKey(Author, on_delete=models.CASCADE)\n\nfrom django.db import transaction, IntegrityError\n\ndef create_author_and_books_django(author_name, book_titles):\n \"\"\"\n Creates an author and multiple books within a single transaction.\n If any step fails, the entire operation is rolled back.\n \"\"\"\n try:\n # Use the 'atomic' block to ensure the operations are transactional\n with transaction.atomic():\n print(f\"Starting transaction to create {author_name} and {len(book_titles)} books...\")\n\n # Operation 1: Create the Author\n author = Author.objects.create(name=author_name)\n print(f\"Created Author: {author.name} (ID: {author.id})\")\n\n # Operation 2+: Create Books linked to the Author\n for i, title in enumerate(book_titles):\n # Simulate a potential error for demonstration (e.g., validation fails)\n if \"Invalid\" in title:\n print(f\"Simulating error for book '{title}'. This will trigger rollback.\")\n raise ValueError(f\"Invalid book title encountered: {title}\")\n\n Book.objects.create(title=title, author=author)\n print(f\"Created Book: '{title}' for {author.name}\")\n\n # If the block finishes without exceptions, the transaction is committed\n print(\"Transaction successful! Changes committed.\")\n\n except (ValueError, IntegrityError) as e:\n # If an exception occurs within the 'atomic' block, the transaction is rolled back\n print(f\"An error occurred: {e}. Transaction rolled back.\")\n # Note: Any partial creations within the block are undone.\n\n\n# --- Example Usage (Requires Django setup) ---\n# Assume you have run 'manage.py migrate' and can import your models\nfrom myapp.models import Author, Book # Adjust import based on your app name\n\n# Example 1: Success\nprint(\"\\n--- Running successful scenario ---\")\ncreate_author_and_books_django(\"Jane Austen\", [\"Pride and Prejudice\", \"Sense and Sensibility\"])\nprint(\"\\n--- Database State After Success ---\")\nprint(\"Authors:\", list(Author.objects.values_list('name', flat=True)))\nprint(\"Books:\", list(Book.objects.values_list('title', flat=True)))\n\n\n```\n\n**SQLAlchemy**:\n\nIn SQLAlchemy, sessions manage transactions. The recommended way to handle transactions is using the session as a context manager (with session). This automatically handles the commit if the block completes successfully and performs a rollback if an exception occurs.\n\n```python\n\n# Assume these models are defined using SQLAlchemy Declarative Base\nfrom sqlalchemy import create_engine, Column, Integer, String, ForeignKey\nfrom sqlalchemy.orm import sessionmaker, relationship\nfrom sqlalchemy.ext.declarative import declarative_base\n\nBase = declarative_base()\n\nclass Author(Base):\n __tablename__ = 'authors_sqla' # Use different table names if mixing ORMs in one DB\n id = Column(Integer, primary_key=True)\n name = Column(String)\n books = relationship(\"Book\", back_populates=\"author\")\n\nclass Book(Base):\n __tablename__ = 'books_sqla'\n id = Column(Integer, primary_key=True)\n title = Column(String)\n author_id = Column(Integer, ForeignKey('authors_sqla.id'))\n author = relationship(\"Author\", back_populates=\"books\")\n\n# --- Example Setup (needed to run SQLAlchemy code) ---\nengine = create_engine('sqlite:///:memory:') # Or your actual database URL\nBase.metadata.create_all(engine) # Create tables based on models\nSessionLocal = sessionmaker(bind=engine) # Configure a session factory\n\nfrom sqlalchemy.exc import SQLAlchemyError # Import specific exceptions as needed\n\ndef create_author_and_books_sqla(session, author_name, book_titles):\n \"\"\"\n Creates an author and multiple books within a SQLAlchemy transaction\n managed by the session context.\n \"\"\"\n try:\n # The 'with session:' context manager handles the transaction:\n # It calls session.begin(), session.commit() on success,\n # and session.rollback() on exception.\n with session: # Start a transaction\n print(f\"Starting SQLAlchemy session/transaction to create {author_name} and {len(book_titles)} books...\")\n\n # Operation 1: Create the Author\n author = Author(name=author_name)\n session.add(author)\n # You might need session.flush() here if subsequent operations\n # immediately need the author.id before commit, but for relationship\n # assignment like book.author = author, it's often handled by SA.\n session.flush() # Get the author's ID assigned if needed\n\n print(f\"Added Author: {author.name} (ID will be assigned on commit or flush)\")\n\n # Operation 2+: Create Books linked to the Author\n for i, title in enumerate(book_titles):\n # Simulate a potential error\n if \"Invalid\" in title:\n print(f\"Simulating error for book '{title}'. This will trigger rollback.\")\n raise ValueError(f\"Invalid book title encountered: {title}\")\n\n book = Book(title=title, author=author) # Link using the Author object\n session.add(book)\n print(f\"Added Book: '{title}' for {author.name}\")\n\n\n # If the block finishes without exceptions, session.commit() is called automatically\n print(\"Transaction successful! Changes committed.\")\n\n except (ValueError, SQLAlchemyError) as e:\n # If an exception occurs, session.rollback() is called automatically by the 'with' block\n print(f\"An error occurred: {e}. Transaction rolled back.\")\n # Note: The session might be in an invalid state after rollback;\n # it's often best to close and get a new one for subsequent operations.\n\n# --- Example Usage (Requires SQLAlchemy setup above) ---\nfrom your_models_file import Author, Book, SessionLocal # Adjust import\nsession = SessionLocal() # Get a new session instance\n\nExample 1: Success\nprint(\"\\n--- Running successful scenario (SQLA) ---\")\ncreate_author_and_books_sqla(session, \"J.R.R. Tolkien\", [\"The Hobbit\", \"The Fellowship of the Ring\"])\nsession.close() # Always close the session\n\n```" } ]