Understand and use safety settings

You can use safety settings to adjust the likelihood of getting responses that may be considered harmful. By default, safety settings block content with medium and/or high probability of being unsafe content across all dimensions.

Jump to Gemini safety settings Jump to Imagen safety settings

Safety settings for Gemini models

Learn more about safety settings for Gemini models in the Google Cloud documentation.

Swift

You configure SafetySettings when you create a GenerativeModel instance.

Example with one safety setting:

// ...

let model = vertex.generativeModel(
  modelName: "GEMINI_MODEL_NAME",
  safetySettings: [
    SafetySetting(harmCategory: .harassment, threshold: .blockOnlyHigh)
  ]
)

// ...

Example with multiple safety settings:

// ...

let harassmentSafety = SafetySetting(harmCategory: .harassment, threshold: .blockOnlyHigh)
let hateSpeechSafety = SafetySetting(harmCategory: .hateSpeech, threshold: .blockMediumAndAbove)

let model = vertex.generativeModel(
  modelName: "GEMINI_MODEL_NAME",
  safetySettings: [harassmentSafety, hateSpeechSafety]
)

// ...

Kotlin

You configure SafetySettings when you create a GenerativeModel instance.

Example with one safety setting:

import com.google.firebase.vertexai.type.HarmBlockThreshold
import com.google.firebase.vertexai.type.HarmCategory
import com.google.firebase.vertexai.type.SafetySetting

val generativeModel = Firebase.vertexAI.generativeModel(
    modelName = "GEMINI_MODEL_NAME",
    safetySettings = listOf(
        SafetySetting(HarmCategory.HARASSMENT, HarmBlockThreshold.ONLY_HIGH)
    )
)

// ...

Example with multiple safety settings:

import com.google.firebase.vertexai.type.HarmBlockThreshold
import com.google.firebase.vertexai.type.HarmCategory
import com.google.firebase.vertexai.type.SafetySetting

val harassmentSafety = SafetySetting(HarmCategory.HARASSMENT, HarmBlockThreshold.ONLY_HIGH)
val hateSpeechSafety = SafetySetting(HarmCategory.HATE_SPEECH, HarmBlockThreshold.MEDIUM_AND_ABOVE)

val generativeModel = Firebase.vertexAI.generativeModel(
    modelName = "GEMINI_MODEL_NAME",
    safetySettings = listOf(harassmentSafety, hateSpeechSafety)
)

// ...

Java

You configure SafetySettings when you create a GenerativeModel instance.

SafetySetting harassmentSafety = new SafetySetting(HarmCategory.HARASSMENT,
    HarmBlockThreshold.ONLY_HIGH);

GenerativeModel gm = FirebaseVertexAI.getInstance().generativeModel(
    "GEMINI_MODEL_NAME",
    /* generationConfig is optional */ null,
    Collections.singletonList(harassmentSafety)
);

GenerativeModelFutures model = GenerativeModelFutures.from(gm);

// ...

Example with multiple safety settings:

SafetySetting harassmentSafety = new SafetySetting(HarmCategory.HARASSMENT,
    HarmBlockThreshold.ONLY_HIGH);

SafetySetting hateSpeechSafety = new SafetySetting(HarmCategory.HATE_SPEECH,
    HarmBlockThreshold.MEDIUM_AND_ABOVE);

GenerativeModel gm = FirebaseVertexAI.getInstance().generativeModel(
    "GEMINI_MODEL_NAME",
    /* generationConfig is optional */ null,
    List.of(harassmentSafety, hateSpeechSafety)
);

GenerativeModelFutures model = GenerativeModelFutures.from(gm);

// ...

Web

You configure SafetySettings when you create a GenerativeModel instance.

Example with one safety setting:

import { HarmBlockThreshold, HarmCategory, getGenerativeModel } from "firebase/vertexai";

// ...

const safetySettings = [
  {
    category: HarmCategory.HARM_CATEGORY_HARASSMENT,
    threshold: HarmBlockThreshold.BLOCK_ONLY_HIGH,
  },
];

const model = getGenerativeModel(vertex, { model: "GEMINI_MODEL_NAME", safetySettings });

// ...

Example with multiple safety settings:

import { HarmBlockThreshold, HarmCategory, getGenerativeModel } from "firebase/vertexai";

// ...

const safetySettings = [
  {
    category: HarmCategory.HARM_CATEGORY_HARASSMENT,
    threshold: HarmBlockThreshold.BLOCK_ONLY_HIGH,
  },
  {
    category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
    threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
  },
];

const model = getGenerativeModel(vertex, { model: "GEMINI_MODEL_NAME", safetySettings });

// ...

Dart

You configure SafetySettings when you create a GenerativeModel instance.

Example with one safety setting:

// ...

final safetySettings = [
  SafetySetting(HarmCategory.harassment, HarmBlockThreshold.high)
];
final model = FirebaseVertexAI.instance.generativeModel(
  model: 'GEMINI_MODEL_NAME',
  safetySettings: safetySettings,
);

// ...

Example with multiple safety settings:

// ...

final safetySettings = [
  SafetySetting(HarmCategory.harassment, HarmBlockThreshold.high),
  SafetySetting(HarmCategory.hateSpeech, HarmBlockThreshold.high),
];
final model = FirebaseVertexAI.instance.generativeModel(
  model: 'GEMINI_MODEL_NAME',
  safetySettings: safetySettings,
);

// ...

Safety settings for Imagen models

Learn about all the supported safety settings and their available values for Imagen models in the Google Cloud documentation.

Swift

You configure ImagenSafetySettings when you create an ImagenModel instance.

// Initialize the Vertex AI service
let vertex = VertexAI.vertexAI()

// Create an `ImagenModel` instance
let model = vertex.imagenModel(
  modelName: "IMAGEN_MODEL_NAME",
  // Configure image generation safety settings for the model
  safetySettings: ImagenSafetySettings(
    safetyFilterLevel: .blockLowAndAbove,
    personFilterLevel: .allowAdult
  )
)

// ...

Kotlin

You configure ImagenSafetySettings when you create an ImagenModel instance.

// Create an `ImagenModel` instance
val imagenModel = Firebase.vertexAI.imagenModel(
  modelName = "IMAGEN_MODEL_NAME",
  // Configure image generation safety settings for the model
  safetySettings = ImagenSafetySettings(
    safetyFilterLevel = ImagenSafetyFilterLevel.BLOCK_LOW_AND_ABOVE,
    personFilterLevel = ImagenPersonFilterLevel.BLOCK_ALL
  )
)

// ...

Java

You configure ImagenSafetySettings when you create an ImagenModel instance.

// Create an `ImagenModel` instance
ImagenModel imagenModel =
    FirebaseVertexAI.getInstance().imagenModel(
            /* modelName */ "IMAGEN_MODEL_NAME",
            /* imageGenerationConfig */ null);
ImagenModelFutures model = ImagenModelFutures.from(imagenModel);

// ...

Web

You configure ImagenSafetySettings when you create an ImagenModel instance.

// Create an `ImagenModel` instance
const imagenModel = getImagenModel(
  vertexAI,
  {
    model: "IMAGEN_MODEL_NAME",
    // Configure image generation safety settings for the model
    safetySettings: {
      safetyFilterLevel: ImagenSafetyFilterLevel.BLOCK_LOW_AND_ABOVE,
      personFilterLevel: ImagenPersonFilterLevel.ALLOW_ADULT,
    }
  }
);

// ...

Dart

You configure ImagenSafetySettings when you create an ImagenModel instance.

// Create an `ImagenModel` instance
var model = FirebaseVertexAI.instance.imagenModel(
  model: 'IMAGEN_MODEL_NAME',
  // Configure image generation safety settings for the model
  safetySettings: ImagenSafetySettings(
    ImagenSafetyFilterLevel.blockLowAndAbove,
    ImagenPersonFilterLevel.allowAdult,
  ),
);

// ...

Other options to control content generation

  • Learn more about prompt design so that you can influence the model to generate output specific to your needs.
  • Configure model parameters to control how the model generates a response. For Gemini models, these parameters include max output tokens, temperature, topK, and topP. For Imagen models, these include aspect ratio, person generation, watermarking, etc.
  • Set system instructions to steer the behavior of the model. This feature is like a "preamble" that you add before the model gets exposed to any further instructions from the end user.
  • Pass a response schema along with the prompt to specify a specific output schema. This feature is most commonly used when generating JSON output, but it can also be used for classification tasks (like when you want the model to use specific labels or tags).