Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
# NDArray Library (Java)

![CI](https://github.com/sMouaad/DevOps-Project/actions/workflows/ci.yml/badge.svg)
![Docker](https://github.com/sMouaad/DevOps-Project/actions/workflows/docker.yml/badge.svg)
[![CI](https://github.com/sMouaad/DevOps-Project/actions/workflows/ci.yml/badge.svg)](https://github.com/sMouaad/DevOps-Project/actions/workflows/ci.yml)
[![Docker](https://github.com/sMouaad/DevOps-Project/actions/workflows/docker.yml/badge.svg)](https://github.com/sMouaad/DevOps-Project/actions/workflows/docker.yml)
[![Pages](https://github.com/sMouaad/DevOps-Project/actions/workflows/deploy-site.yml/badge.svg)](https://github.com/sMouaad/DevOps-Project/actions/workflows/deploy-site.yml)
[![Publish SNAPSHOT](https://github.com/sMouaad/DevOps-Project/actions/workflows/publish-snapshot.yml/badge.svg)](https://github.com/sMouaad/DevOps-Project/actions/workflows/publish-snapshot.yml)
[![Docs](https://img.shields.io/badge/docs-live-success)](https://smouaad.github.io/DevOps-Project/)
[![Java 17](https://img.shields.io/badge/java-17-blue)](https://adoptium.net/)
[![Maven](https://img.shields.io/badge/build-maven-C71A36)](https://maven.apache.org/)

Small NumPy-inspired Java library for ndarray operations, developed as a DevOps team project.

Expand All @@ -24,6 +29,7 @@ Current snapshot date: 2026-04-04.
- 2D ndarray support with matrix validation.
- Creation helpers: `array`, `zeros`, `arange`.
- Addition operations: `add` and `addInPlace` with strict shape checks.
- Optional extensions: scalar addition and `sum()` reduction.
- `reshape(int... shape)` with size consistency validation.
- NumPy-like string display for 1D arrays (including ellipsis for large arrays).
- Defensive copy behavior on constructors and exports.
Expand All @@ -38,6 +44,8 @@ NdArray c = NdArray.arange(0f, 6f, 2f);
NdArray m = NdArray.array(new float[][] {{1f, 2f}, {3f, 4f}});
NdArray sum = m.add(NdArray.array(new float[][] {{10f, 10f}, {10f, 10f}}));
NdArray reshaped = NdArray.arange(6f).reshape(2, 3);
NdArray shifted = a.add(10f);
float total = shifted.sum();
```

## Tech Stack and Tooling Choices
Expand Down Expand Up @@ -156,14 +164,30 @@ The Docker workflow (`.github/workflows/docker.yml`):
- Uses multi-stage build for minimal image size
- Caches layers with GitHub Actions cache

## Cloud Deployment

We use Terraform + Ansible to deploy the container to GCP. Spins up a free-tier e2-micro VM with Docker.

```bash
cd infrastructure/scripts
./setup-gcp.sh # one-time setup
./deploy.sh # creates VM and deploys container
./destroy.sh # tear down when done
```

See [infrastructure/README.md](infrastructure/README.md) for details.

## Current Status vs Mandatory Scope

Implemented:

- 1D and 2D ndarray support.
- Creation functions (`array`, `zeros`, `arange`).
- Addition operations (`add`, `addInPlace`).
- Scalar addition and `sum()` reduction.
- Reshape with validation.
- Docker containerization with CI/CD pipeline.
- Infrastructure as Code for GCP deployment.

In progress / next:

Expand Down
29 changes: 29 additions & 0 deletions infrastructure/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
*.tfstate
*.tfstate.*
*.tfvars
!terraform.tfvars.example
.terraform/
.terraform.lock.hcl
tfplan
*.tfplan
crash.log
override.tf
override.tf.json
*_override.tf
*_override.tf.json

*.pem
*.pub
id_rsa*
ssh_key*

credentials.json
*-credentials.json
*-key.json
service-account*.json

*.retry
ansible/inventory/hosts.yml

.env
.env.*
122 changes: 122 additions & 0 deletions infrastructure/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# IaC - GCP Deployment

Terraform + Ansible setup to deploy our ndarray container on GCP.

## What we're deploying

- 1 VM (e2-micro, free tier) on europe-west1
- Custom VPC with firewall for SSH/HTTP
- Docker container running the ndarray demo

## Setup

You need Terraform, Ansible, and gcloud CLI installed. Check with:
```bash
terraform --version && ansible --version && gcloud --version
```

### GCP account

Create a project on https://console.cloud.google.com and grab the project ID.

```bash
gcloud auth login
gcloud config set project YOUR_PROJECT_ID
```

## Deploy

```bash
cd infrastructure/scripts

# first time only - enables APIs, creates service account, generates keys
./setup-gcp.sh

# set creds
export GOOGLE_APPLICATION_CREDENTIALS="$(pwd)/../terraform/credentials.json"

# deploy everything
./deploy.sh
```

The deploy script runs terraform to create the VM, then ansible to install docker and start the container.

### Check it worked

```bash
ssh -i ../ssh_key ansible@<VM_IP>
docker ps
docker logs ndarray-demo
```

## Manual way

If you want more control:

```bash
# terraform
cd infrastructure/terraform
terraform init
terraform plan
terraform apply
terraform output vm_external_ip

# ansible
cd ../ansible
ansible-playbook playbooks/setup.yml
ansible-playbook playbooks/deploy.yml
```

## Tear down

Don't forget to destroy when you're done:
```bash
./destroy.sh
# or: terraform destroy
```

## Files

```
infrastructure/
├── terraform/ # VM, VPC, firewall definitions
├── ansible/ # playbooks + roles for docker/app setup
└── scripts/ # setup, deploy, destroy helpers
```

## Config

Edit `terraform/terraform.tfvars`:
```hcl
project_id = "your-project-id"
project_name = "ndarray"
region = "europe-west1"
zone = "europe-west1-b"
machine_type = "e2-micro"
ssh_user = "ansible"
```

## Common issues

**Can't SSH?**
```bash
gcloud compute instances list # check VM is running
ssh -i infrastructure/ssh_key -o StrictHostKeyChecking=no ansible@<VM_IP>
```

**Terraform state messed up?**
```bash
terraform refresh
terraform taint google_compute_instance.vm
terraform apply
```

**Container not starting?**
```bash
sudo docker ps -a
sudo docker logs ndarray-demo
```

## Costs

e2-micro is free tier (1/month). Static IP has a small cost when unattached. Just run destroy.sh when not using it.
18 changes: 18 additions & 0 deletions infrastructure/ansible/ansible.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[defaults]
inventory = inventory/hosts.yml
roles_path = roles
remote_user = ansible
private_key_file = ../ssh_key
host_key_checking = False
retry_files_enabled = False
interpreter_python = auto_silent

[privilege_escalation]
become = True
become_method = sudo
become_user = root
become_ask_pass = False

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no
pipelining = True
9 changes: 9 additions & 0 deletions infrastructure/ansible/inventory/hosts.yml.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
all:
children:
app_servers:
hosts:
ndarray-vm:
ansible_host: "{{ vm_ip }}"
ansible_user: ansible
ansible_ssh_private_key_file: "../ssh_key"
12 changes: 12 additions & 0 deletions infrastructure/ansible/playbooks/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
# Deploy playbook - Builds and runs the application container
- name: Deploy Application
hosts: app_servers
become: yes

vars:
app_name: ndarray-demo
app_version: latest

roles:
- app
16 changes: 16 additions & 0 deletions infrastructure/ansible/playbooks/setup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
# Setup playbook - Configures VM with Docker
- name: Configure VM
hosts: app_servers
become: yes

tasks:
- name: Wait for system to be ready
wait_for_connection:
timeout: 300

- name: Gather facts
setup:

roles:
- docker
4 changes: 4 additions & 0 deletions infrastructure/ansible/roles/app/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
app_name: ndarray-demo
app_version: latest
container_ports: []
55 changes: 55 additions & 0 deletions infrastructure/ansible/roles/app/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
- name: Create app directory
file:
path: /opt/ndarray
state: directory
mode: '0755'

- name: Copy Dockerfile
copy:
src: "{{ playbook_dir }}/../../../Dockerfile"
dest: /opt/ndarray/Dockerfile
mode: '0644'

- name: Copy pom.xml
copy:
src: "{{ playbook_dir }}/../../../pom.xml"
dest: /opt/ndarray/pom.xml
mode: '0644'

- name: Copy source code
copy:
src: "{{ playbook_dir }}/../../../src/"
dest: /opt/ndarray/src/
mode: '0644'
directory_mode: '0755'

- name: Build Docker image
community.docker.docker_image:
name: "{{ app_name }}"
tag: "{{ app_version }}"
source: build
build:
path: /opt/ndarray
pull: yes
state: present
register: docker_build

- name: Remove existing container if present
community.docker.docker_container:
name: "{{ app_name }}"
state: absent
ignore_errors: yes

- name: Run application container
community.docker.docker_container:
name: "{{ app_name }}"
image: "{{ app_name }}:{{ app_version }}"
state: started
restart_policy: unless-stopped
ports: "{{ container_ports | default([]) }}"
register: container_result

- name: Display container status
debug:
msg: "Container '{{ app_name }}' is running with ID: {{ container_result.container.Id[:12] }}"
49 changes: 49 additions & 0 deletions infrastructure/ansible/roles/docker/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
- name: Install Docker dependencies
apt:
name:
- apt-transport-https
- ca-certificates
- curl
- gnupg
- lsb-release
- python3-pip
state: present
update_cache: yes

- name: Add Docker GPG key
apt_key:
url: https://download.docker.com/linux/ubuntu/gpg
state: present

- name: Add Docker repository
apt_repository:
repo: "deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable"
state: present

- name: Install Docker
apt:
name:
- docker-ce
- docker-ce-cli
- containerd.io
- docker-compose-plugin
state: present
update_cache: yes

- name: Start and enable Docker service
systemd:
name: docker
state: started
enabled: yes

- name: Add user to docker group
user:
name: "{{ ansible_user }}"
groups: docker
append: yes

- name: Install Docker Python SDK
pip:
name: docker
state: present
Loading
Loading