# Custom Resources & Operators Guide

## 🎯 **Overview**

This guide explains **Custom Resources (CRs)**, **Custom Resource Definitions (CRDs)**, **Kubernetes Operators**, and how your `freeleaps-devops-reconciler` works as an operator to manage your DevOps infrastructure.

---

## 📊 **What Are Custom Resources?**

### **🔄 CR vs CRD Relationship**

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                              CRD vs CR RELATIONSHIP                         │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                    CUSTOM RESOURCE DEFINITION (CRD)                │   │   │
│  │                                                                     │   │   │
│  │  apiVersion: apiextensions.k8s.io/v1                              │   │   │
│  │  kind: CustomResourceDefinition                                    │   │   │
│  │  metadata:                                                         │   │   │
│  │    name: devopsprojects.freeleaps.com                             │   │   │
│  │  spec:                                                             │   │   │
│  │    group: freeleaps.com                                            │   │   │
│  │    names:                                                          │   │   │
│  │      kind: DevOpsProject                                          │   │   │
│  │      plural: devopsprojects                                       │   │   │
│  │    scope: Namespaced                                              │   │   │
│  │    versions:                                                      │   │   │
│  │      - name: v1alpha1                                             │   │   │
│  │        schema:                                                    │   │   │
│  │          # Schema definition...                                   │   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │   │
│                                    │                                        │   │
│                                    ▼                                        │   │
│  ┌─────────────────────────────────────────────────────────────────────┐   │   │
│  │                    CUSTOM RESOURCE (CR)                            │   │   │
│  │                                                                     │   │   │
│  │  apiVersion: freeleaps.com/v1alpha1                               │   │   │
│  │  kind: DevOpsProject                                              │   │   │
│  │  metadata:                                                        │   │   │
│  │    name: my-project                                              │   │   │
│  │    namespace: freeleaps-devops-system                            │   │   │
│  │  spec:                                                           │   │   │
│  │    projectName: "My Awesome Project"                            │   │   │
│  │    projectId: "my-awesome-project"                              │   │   │
│  │    git:                                                          │   │   │
│  │      url: "https://github.com/myorg/myproject"                   │   │   │
│  │      branch: "main"                                              │   │   │
│  │    registry:                                                      │   │   │
│  │      url: "https://harbor.example.com"                          │   │   │
│  │      project: "myproject"                                        │   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │   │
└─────────────────────────────────────────────────────────────────────────────┘
```

### **🎯 Why Custom Resources?**

```yaml
# Instead of managing multiple resources manually:
# - Namespace
# - ServiceAccount
# - Role/RoleBinding
# - ConfigMap
# - Secret
# - Deployment
# - Service
# - Ingress

# You can create ONE custom resource:
apiVersion: freeleaps.com/v1alpha1
kind: DevOpsProject
metadata:
  name: my-project
spec:
  projectName: "My Project"
  projectId: "my-project"
  git:
    url: "https://github.com/myorg/myproject"
    branch: "main"
  registry:
    url: "https://harbor.example.com"
    project: "myproject"
```

---

## 🏭 **Your DevOps Reconciler Architecture**

### **📊 Reconciler vs DevOps Repo Relationship**

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                    RECONCILER + DEVOPS ARCHITECTURE                          │
│                                                                             │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │                    FRELEAPS.COM PLATFORM                           │   │   │
│  │                                                                     │   │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │   │
│  │  │   User      │  │   Project   │  │   Git        │  │   Registry  │   │   │
│  │  │   Creates   │  │   Manager   │  │   Webhook    │  │   Manager   │   │   │
│  │  │   Project   │  │   Creates    │  │   Triggers   │  │   Creates   │   │   │
│  │  │             │  │   DevOps    │  │   Event      │  │   Repo      │   │   │
│  │  │             │  │   Project   │  │               │  │             │   │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │   │
│                                    │                                        │   │
│                                    ▼                                        │   │
│  ┌─────────────────────────────────────────────────────────────────────┐   │   │
│  │                    RABBITMQ MESSAGE QUEUE                          │   │   │
│  │                                                                     │   │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │   │
│  │  │DevOpsInit   │  │DevOpsReconcile│  │DevOpsDeploy │  │DevOpsDelete │   │   │
│  │  │Event        │  │Event         │  │Event        │  │Event        │   │   │
│  │  │             │  │              │  │             │  │             │   │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │   │
│                                    │                                        │   │
│                                    ▼                                        │   │
│  ┌─────────────────────────────────────────────────────────────────────┐   │   │
│  │                FRELEAPS-DEVOPS-RECONCILER (OPERATOR)                │   │   │
│  │                                                                     │   │   │
│  │  ┌─────────────────────────────────────────────────────────────┐   │   │   │
│  │  │                    CONTROLLERS                              │   │   │   │
│  │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────┐ │   │   │   │
│  │  │  │DevOpsProject│  │ArgoSettings │  │Jenkins      │  │...  │ │   │   │   │
│  │  │  │Controller   │  │Controller   │  │Settings     │  │     │ │   │   │   │
│  │  │  │             │  │             │  │Controller   │  │     │ │   │   │   │
│  │  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────┘ │   │   │   │
│  │  └─────────────────────────────────────────────────────────────┘   │   │   │
│  │                                    │                                │   │   │
│  │                                    ▼                                │   │   │
│  │  ┌─────────────────────────────────────────────────────────────┐   │   │   │
│  │  │                    CUSTOM RESOURCES                         │   │   │   │
│  │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────┐ │   │   │   │
│  │  │  │DevOpsProject│  │ArgoSettings │  │Jenkins      │  │...  │ │   │   │   │
│  │  │  │CR           │  │CR           │  │Settings     │  │     │ │   │   │   │
│  │  │  │             │  │             │  │CR           │  │     │ │   │   │   │
│  │  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────┘ │   │   │   │
│  │  └─────────────────────────────────────────────────────────────┘   │   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │   │
│                                    │                                        │   │
│                                    ▼                                        │   │
│  ┌─────────────────────────────────────────────────────────────────────┐   │   │
│  │                    KUBERNETES RESOURCES                             │   │   │
│  │                                                                     │   │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐   │   │
│  │  │ArgoCD       │  │Jenkins      │  │Harbor       │  │Namespaces    │   │   │
│  │  │Applications │  │Pipelines    │  │Repositories │  │Services      │   │   │
│  │  │Projects     │  │Jobs         │  │Credentials  │  │Deployments   │   │   │
│  │  │             │  │             │  │             │  │Ingresses     │   │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘   │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

## 🔧 **Your Custom Resources**

### **1. DevOpsProject CRD**

```yaml
# 🏭 ACTUAL CRD FROM YOUR CODEBASE
# freeleaps-devops-reconciler/deploy/crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: devopsprojects.freeleaps.com
spec:
  group: freeleaps.com
  scope: Namespaced
  names:
    kind: DevOpsProject
    plural: devopsprojects
    shortNames: [dop, dops]
  versions:
    - name: v1alpha1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          required: ['spec']
          properties:
            spec:
              type: object
              required:
                - projectName
                - projectId
                - git
                - registry
                - environments
              properties:
                projectName:
                  type: string
                  description: "Human readable project name"
                projectId:
                  type: string
                  description: "Unique project identifier"
                  pattern: "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$"
                git:
                  type: object
                  required: [url, branch]
                  properties:
                    url:
                      type: string
                      description: "Git repository URL"
                    branch:
                      type: string
                      description: "Default git branch"
                      default: "main"
                registry:
                  type: object
                  required: [url, project]
                  properties:
                    url:
                      type: string
                      description: "Container registry URL"
                    project:
                      type: string
                      description: "Registry project name"
```

### **2. DevOpsProject CR Example**

```yaml
# 🏭 ACTUAL CR EXAMPLE
apiVersion: freeleaps.com/v1alpha1
kind: DevOpsProject
metadata:
  name: magicleaps-frontend
  namespace: freeleaps-devops-system
  labels:
    app.kubernetes.io/name: magicleaps-frontend
    app.kubernetes.io/instance: magicleaps-frontend
spec:
  projectName: "Magicleaps Frontend"
  projectId: "magicleaps-frontend"
  git:
    url: "https://github.com/freeleaps/magicleaps-frontend"
    branch: "main"
    credentialsRef:
      name: "github-credentials"
      namespace: "freeleaps-devops-system"
  registry:
    url: "https://harbor.freeleaps.mathmast.com"
    project: "magicleaps"
    credentialsRef:
      name: "harbor-credentials"
      namespace: "freeleaps-devops-system"
  environments:
    - name: "production"
      domain: "magicleaps.mathmast.com"
      replicas: 3
    - name: "alpha"
      domain: "alpha.magicleaps.mathmast.com"
      replicas: 1
```

### **3. Other Custom Resources**

```yaml
# 🏭 YOUR COMPLETE CRD SET
# From freeleaps-devops-reconciler/docs/design/one-click-deployment.md

# DevOpsProject - Main project configuration
# ArgoSettings - ArgoCD settings for the DevOpsProject
# JenkinsSettings - Jenkins settings and generated pipelines
# ContainerRegistry - Container registry information
# ContainerImage - Every image manufactured by Jenkins pipeline
# DeploymentRecord - Track deployment records
# GitCredential - Git repository credentials
# IngressResource - Ingress configuration
```

---

## 🤖 **How Your Operator Works**

### **🔄 Reconciliation Loop**

```python
# 🏭 ACTUAL CODE FROM YOUR RECONCILER
# freeleaps-devops-reconciler/reconciler/controllers/devops_projects/controller.py

@kopf.on.create(group=consts.GROUP, version=consts.VERSION, kind=consts.DEVOPS_PROJECT_KIND)
def on_devops_proj_created(name: str, namespace: Optional[str], body: Body, logger: Logger, **kwargs):
    logger.info(f"Newly created DevOpsProject resource and named {name} in namespace {namespace}, start to reconciling...")
    
    devops_proj = DevOpsProject(body)
    
    try:
        devops_proj.parse_spec()
        devops_proj.get_spec().validate(logger)
    except SpecError as e:
        devops_proj.update_status({
            'devopsProject': {
                'status': DevOpsProjectDiagStatus.INVALID.value,
                'synced': False,
                'ready': False,
                'lastProbeTime': isotime(),
            }
        })
        devops_proj.error(action='CreateDevOpsProject',
                          reason='InvalidSpecArgument', msg=str(e))
        raise kopf.TemporaryError(f"Error found in DevOpsProject spec: {e}")
    
    # Create resource manager and handle the project
    resource_manager = DevOpsProjectResourceManager(namespace, logger)
    # ... implementation details
```

### **📊 Event Flow**

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                              EVENT FLOW                                     │
│                                                                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐      │
│  │   User      │  │   RabbitMQ  │  │   Operator  │  │   Kubernetes │      │
│  │   Action    │  │   Message   │  │   Controller│  │   Resources  │      │
│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘      │
│         │                │                │                │                │
│         │ 1. Create      │                │                │                │
│         │ Project       │                │                │                │
│         │───────────────▶│                │                │                │
│         │                │ 2. DevOpsInit  │                │                │
│         │                │ Event          │                │                │
│         │                │───────────────▶│                │                │
│         │                │                │ 3. Create CR   │                │
│         │                │                │───────────────▶│                │
│         │                │                │                │ 4. CR Created │
│         │                │                │                │◀───────────────│
│         │                │                │ 5. Reconcile  │                │
│         │                │                │◀───────────────│                │
│         │                │                │ 6. Create     │                │
│         │                │                │ Resources     │                │
│         │                │                │───────────────▶│                │
│         │                │                │                │ 7. Resources  │
│         │                │                │                │ Ready         │
│         │                │                │                │◀───────────────│
│         │                │                │ 8. Update      │                │
│         │                │                │ Status         │                │
│         │                │                │◀───────────────│                │
└─────────────────────────────────────────────────────────────────────────────┘
```

---

## 🎯 **Understanding the Relationship**

### **📊 Reconciler vs DevOps Repo**

| Component | Purpose | Location | Responsibility |
|-----------|---------|----------|----------------|
| **freeleaps-devops-reconciler** | **Kubernetes Operator** | `freeleaps-ops/freeleaps-devops-reconciler/` | Watches CRs, creates K8s resources |
| **freeleaps.com Platform** | **Business Logic** | `freeleaps-service-hub/` | User interface, project management |
| **RabbitMQ** | **Message Queue** | Infrastructure | Event communication |
| **ArgoCD** | **GitOps** | `freeleaps-ops/cluster/manifests/` | Application deployment |
| **Jenkins** | **CI/CD** | Infrastructure | Pipeline execution |

### **🔄 How They Work Together**

```yaml
# 1. User creates project on freeleaps.com
# 2. Platform sends DevOpsInit event to RabbitMQ
# 3. Reconciler receives event and creates DevOpsProject CR
# 4. Reconciler watches CR and creates:
#    - ArgoCD Application
#    - Jenkins Pipeline
#    - Harbor Repository
#    - Namespace and RBAC
# 5. ArgoCD deploys the application
# 6. Jenkins runs the pipeline
```

---

## 🔧 **Practical Examples**

### **1. Creating a DevOpsProject**

```bash
# Create a DevOpsProject CR
kubectl apply -f - <<EOF
apiVersion: freeleaps.com/v1alpha1
kind: DevOpsProject
metadata:
  name: my-new-project
  namespace: freeleaps-devops-system
spec:
  projectName: "My New Project"
  projectId: "my-new-project"
  git:
    url: "https://github.com/myorg/myproject"
    branch: "main"
  registry:
    url: "https://harbor.example.com"
    project: "myproject"
  environments:
    - name: "production"
      domain: "myproject.example.com"
      replicas: 2
EOF

# Check the CR status
kubectl get devopsprojects -n freeleaps-devops-system
kubectl describe devopsproject my-new-project -n freeleaps-devops-system
```

### **2. Monitoring the Operator**

```bash
# Check operator logs
kubectl logs -n freeleaps-devops-system deployment/freeleaps-devops-reconciler

# Check CR status
kubectl get devopsprojects --all-namespaces
kubectl get argosettings --all-namespaces
kubectl get jenkinssettings --all-namespaces

# Check created resources
kubectl get applications -n freeleaps-devops-system
kubectl get namespaces | grep my-new-project
```

### **3. Troubleshooting**

```bash
# Check CRD installation
kubectl get crd | grep freeleaps.com

# Check operator events
kubectl get events -n freeleaps-devops-system --sort-by='.lastTimestamp'

# Check resource creation
kubectl get all -n my-new-project
kubectl describe devopsproject my-new-project -n freeleaps-devops-system
```

---

## 🎯 **Best Practices**

### **1. CRD Design**

```yaml
# ✅ DO: Use clear, descriptive names
apiVersion: freeleaps.com/v1alpha1
kind: DevOpsProject  # Clear, descriptive

# ❌ DON'T: Use generic names
kind: Project  # Too generic
```

### **2. Validation**

```yaml
# ✅ DO: Include validation in CRD
spec:
  openAPIV3Schema:
    type: object
    required: ['spec']
    properties:
      spec:
        type: object
        required: ['projectName', 'projectId']
        properties:
          projectId:
            pattern: "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$"
```

### **3. Status Management**

```yaml
# ✅ DO: Include status in CR
status:
  conditions:
    - type: Ready
      status: "True"
      reason: "ReconciliationSucceeded"
      message: "All resources created successfully"
    - type: Synced
      status: "True"
      reason: "ReconciliationSucceeded"
      message: "Spec has been reconciled"
```

### **4. Error Handling**

```python
# ✅ DO: Proper error handling
try:
    # Create resources
    resource_manager.create_resources()
except Exception as e:
    # Update status with error
    devops_proj.update_status({
        'status': 'Error',
        'message': str(e)
    })
    raise kopf.TemporaryError(f"Failed to create resources: {e}")
```

---

## 🔍 **Debugging Your Operator**

### **1. Check Operator Status**

```bash
# Check if operator is running
kubectl get pods -n freeleaps-devops-system -l app=freeleaps-devops-reconciler

# Check operator logs
kubectl logs -n freeleaps-devops-system deployment/freeleaps-devops-reconciler -f

# Check CRD installation
kubectl get crd devopsprojects.freeleaps.com
```

### **2. Check CR Status**

```bash
# Check CR status
kubectl get devopsprojects --all-namespaces -o wide

# Check CR events
kubectl describe devopsproject <project-name> -n <namespace>

# Check CR YAML
kubectl get devopsproject <project-name> -n <namespace> -o yaml
```

### **3. Check Created Resources**

```bash
# Check what resources were created
kubectl get all -n <project-namespace>

# Check ArgoCD applications
kubectl get applications -n freeleaps-devops-system

# Check Jenkins pipelines
kubectl get jenkinssettings --all-namespaces
```

---

## 📚 **Next Steps**

### **1. Learn More About Operators**
- [Kubernetes Operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)
- [Kopf Framework](https://kopf.readthedocs.io/) (what your reconciler uses)
- [Operator SDK](https://sdk.operatorframework.io/)

### **2. Understand Your Architecture**
- Study your `freeleaps-devops-reconciler` code
- Understand the event flow from RabbitMQ
- Learn how CRs trigger resource creation

### **3. Extend Your Operator**
- Add new custom resources
- Implement new controllers
- Add validation and error handling

---

**Last Updated**: September 3, 2025
**Version**: 1.0
**Maintainer**: Infrastructure Team