Training Metrics

Once you've trained a model in LandingLens, it's crucial to analyze its performance. Training metrics provide valuable insights into how well your model classifies images within your project.

Retrieving Training Metrics

LandingLens offers an API endpoint to retrieve the training metrics for a specific training session. Here are the details:

The response is a JSON object containing information about the training metrics for different splits (e.g., train, dev, test). Here's a breakdown of the response structure:

{
  "data": [
    {
      "name": "train",  # Split name (e.g., train, dev)
      "classes": [  # Individual class metrics
        {
          "id": 121683,  # Class ID
          "performance": 0.975,  # Performance metric value
          "recall": 0.974,      # Recall metric value
          "precision": 0.975     # Precision metric value
        },
        // ... other classes
      ],
      "metrics": [  # Overall metrics for the split
        {
          "name": "all",      # Metric name (currently "all")
          "performance": 0.975,  # Performance metric value
          "recall": 0.974,      # Recall metric value
          "precision": 0.975     # Precision metric value
        }
      ]
    },
    // ... other splits (if applicable)
    {
      "name": "test",  # May be null if test set wasn't used
      "classes": null,  # May be null if test set wasn't used
      "metrics": null   # May be null if test set wasn't used
    }
  ]
}

Available Training Metrics

Currently, the LandingLens platform provides the following training metrics through the API:

• Performance
• Precision
• Recall

You can learn more about these metrics in Analyze Models.

Retrieving Training Metrics

import requests

# Replace "YOUR_API_KEY" and "YOUR_PROJECT_ID" with the actual values
api_key = "YOUR_API_KEY"
project_id = "YOUR_PROJECT_ID"
training_id = "YOUR_TRAINING_ID"  # Replace with the specific training ID

# Define the API endpoint URL
url = f"https://api.landing.ai/v1/projects/{project_id}/train/{training_id}/results/metrics"

# Set the authorization header
headers = {"apikey": api_key}

# Send the GET request
response = requests.get(url, headers=headers)

# Check for successful response and handle errors
if response.status_code == 200:
    data = response.json()
    print("Training Metrics:")

    # Iterate through splits and access metrics
    for split in data["data"]:
        print(f"\nSplit: {split['name']}")
        for metric in split["metrics"]:
            print(f"{metric['name']}: {metric['performance']:.3f}")  

        # Access individual class metrics (if applicable)
        if split["classes"]:
            print("\nIndividual Class Metrics:")

Confusion Matrix

The confusion matrix is a valuable tool for visualizing and evaluating the performance of your image classification model in LandingLens. It provides insights into how often your model correctly classifies images and where it might be making mistakes.

Retrieving Confusion Matrix

LandingLens offers an API endpoint to retrieve the confusion matrix for a specific training session. Here are the details:

The response is a JSON object containing information about the confusion matrix for different splits (e.g., train, dev). Here's a breakdown of the response structure:

{
  "data": [
    {
      "name": "train",  # Split name (e.g., train, dev)
      "correctPredictions": [  # List of correctly classified images
        {
          "count": 3,               # Number of correctly classified images
          "groundTruthClassId": 121683,  # Ground truth class ID
          "predictedClassId": 121683    # Predicted class ID
        },
        // ... other correct predictions
      ],
      "falseNegativePredictions": [],  # List of false negative predictions (optional)
      "falsePositivePredictions": [],  # List of false positive predictions (optional)
      "misclassifications": []        # List of misclassifications (optional) - might be empty
    },
    // ... other splits (if applicable)
  ]
}

Retrieving Confusion Matrix

import requests

# Replace "YOUR_API_KEY" and "YOUR_PROJECT_ID" with the actual values
api_key = "YOUR_API_KEY"
project_id = "YOUR_PROJECT_ID"
training_id = "YOUR_TRAINING_ID"  # Replace with the specific training ID

# Define the API endpoint URL
url = f"https://api.landing.ai/v1/projects/{project_id}/train/{training_id}/results/confusion-matrix"

# Set the authorization header
headers = {"apikey": api_key}

# Send the GET request
response = requests.get(url, headers=headers)

# Check for successful response and handle errors
if response.status_code == 200:
    data = response.json()
    print("Confusion Matrix:")

    # Iterate through splits and access correct predictions
    for split in data["data"]:
        print(f"\nSplit: {split['name']}")
        print("Correct Predictions:")
        for prediction in split["correctPredictions"]:
            print(f"- Ground Truth Class: {prediction['groundTruthClassId']}, Predicted Class: {prediction['predictedClassId']}, Count: {prediction['count']}")