AWS S3 + Django - File Upload, Download, List, and Delete Example

To handle file uploads, listing, downloading, and deleting files from an AWS S3 bucket in a Django application, you can use the boto3 library (the AWS SDK for Python) in combination with Django's settings and views. 

Below is a guide to set up these operations:

1. Install Required Packages

First, install boto3 and django-storages to integrate AWS S3 with Django:

pip install boto3 django-storages

2. Configure AWS S3 in Django Settings

In your settings.py file, add the following configurations for AWS S3:

import os
from django.conf import settings

AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')  # Store securely, e.g., in environment variables
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = 'your-s3-bucket-name'
AWS_REGION_NAME = 'your-region'  # e.g., 'us-west-2'

AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

Make sure to replace your-s3-bucket-name and your-region with your actual bucket name and region.

3. Uploading Files to S3

For file upload, you can either use Django’s built-in FileField or handle file uploads programmatically. 

Here’s an example using boto3 directly:

import boto3
from django.conf import settings

def upload_to_s3(file, filename):
    s3_client = boto3.client(
        's3',
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION_NAME
    )
    s3_client.upload_fileobj(file, settings.AWS_STORAGE_BUCKET_NAME, filename)
    file_url = f"https://{settings.AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com/{filename}"
    return file_url

To use this function in your Django views:

from django.http import JsonResponse

def upload_view(request):
    if request.method == 'POST' and request.FILES.get('file'):
        file = request.FILES['file']
        filename = file.name
        file_url = upload_to_s3(file, filename)
        return JsonResponse({'file_url': file_url})
    return JsonResponse({'error': 'No file uploaded'}, status=400)

4. Listing Files in S3 Bucket

To list the files in your S3 bucket, you can use the list_objects_v2 method of boto3:

def list_files_in_s3():
    s3_client = boto3.client(
        's3',
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION_NAME
    )
    response = s3_client.list_objects_v2(Bucket=settings.AWS_STORAGE_BUCKET_NAME)
    
    if 'Contents' in response:
        return [obj['Key'] for obj in response['Contents']]
    return []

In your view:

def list_files_view(request):
    files = list_files_in_s3()
    return JsonResponse({'files': files})

5. Downloading Files from S3

To download a file from S3, you can use the download_file method:

def download_from_s3(filename, download_path):
    s3_client = boto3.client(
        's3',
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION_NAME
    )
    s3_client.download_file(settings.AWS_STORAGE_BUCKET_NAME, filename, download_path)

In your view:

from django.http import HttpResponse

def download_file_view(request, filename):
    download_path = '/path/to/save/file'  # Set an appropriate path
    download_from_s3(filename, download_path)
    with open(download_path, 'rb') as file:
        response = HttpResponse(file.read(), content_type="application/octet-stream")
        response['Content-Disposition'] = f'attachment; filename={filename}'
        return response

6. Deleting Files from S3

To delete a file from S3, you can use the delete_object method:

def delete_from_s3(filename):
    s3_client = boto3.client(
        's3',
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION_NAME
    )
    s3_client.delete_object(Bucket=settings.AWS_STORAGE_BUCKET_NAME, Key=filename)

In your view:

def delete_file_view(request, filename):
    delete_from_s3(filename)
    return JsonResponse({'message': 'File deleted successfully'})

7. Example URLs in urls.py

Add Dependencies

Add the required dependencies for Spring Boot and Couchbase in your pom.xml (if using Maven):

from django.urls import path
from .views import upload_view, list_files_view, download_file_view, delete_file_view

urlpatterns = [
    path('upload/', upload_view, name='upload'),
    path('list/', list_files_view, name='list_files'),
    path('download/<str:filename>/', download_file_view, name='download_file'),
    path('delete/<str:filename>/', delete_file_view, name='delete_file'),
]

Summary

• Uploading: Use boto3.client.upload_fileobj() to upload a file. 
• Listing: Use boto3.client.list_objects_v2() to list files. 
• Downloading: Use boto3.client.download_file() to download files. 
• Deleting: Use boto3.client.delete_object() to delete files. 

This setup will allow your Django application to interact with AWS S3 for file upload, listing, download, and delete operations.