Recommendation for implementing quota checks during login and API calls for Gemini Image Generation and Veo in Google Cloud

[pwc-rm]

Hello,

I want to implement quota validation to ensure users or their groups/departments have sufficient quota before they can log in to use Gemini Image Generation or Veo.

The main requirements are:

• Check quota at login and prevent login if there is insufficient quota, with an appropriate error message.
• For logged-in users, check the remaining quota on every request to Gemini Image Generation or Veo APIs.
• Route all such API calls through an API Gateway to centralize quota validation.
• Store and check quota information in a database keyed by user or group/department.
• Update quota usage atomically to avoid race conditions.

Given these requirements and the Google Cloud environment, what would be the recommended best practice for implementing such quota checks? Specifically:

• Is it advisable to implement quota validation as a middleware/service behind the API Gateway?
• Should quota checks be integrated within the API Gateway policies or handled separately?
• Are there any recommended Google Cloud services or architectures to facilitate quota management and validation?

Any references to existing implementations or patterns would be greatly appreciated.
Thank you very much for your advice and guidance!

Best regards,
Raj

[github-actions]

🤖 Hi @pwc-rm, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[ghchinoy]

Adding for reference some notes for quota access

• Gemini models are on Dynamic Shared Quotas - i'm not sure these are queryable in the way you are describing for management; some customers may also have Provisioned Throughput, which makes obtaining this information complex
  "For the default pay-as-you-go model, Vertex AI uses Dynamic Shared Quota, which doesn't have a predefined usage limit. Instead, you get access to a large, shared pool of resources that are dynamically allocated based on real-time availability and demand."

• Quotas in general require a different role than one might want to give this app or a backend for retrieving this info

[FrankMock]

I think we are talking about different things: The quota my colleague is talking about is end user related. Suppose the app is used by multiple departments in a company - or more generalized: multiple groups. There are one or more app managers who control how much budget can be consumed by each department and individual end user in a certain timeframe. For example: the marketing department has a monthly budget of 20,000 € and the HR department has a monthly budget of 10,000 €. These budgets are related to image, video, voice and sound generation (all the content generation possible in the app). Each content generation reduces the available monthly budget. If the budget of the marketing department is fully consumed on the 20th of a month no further generation can take place this month and the app blocks the content generation for all users of the department until the end of the month. In addition the app tracks how much budget is consumed by each department and user to allow an allocation of the content generation costs to the correct department in the finance and controlling systems. The tracking on end user level shall ensure that end users can not misuse the app.
To enable a functionality like this a central budget management module is required that:

• Allows the definition of departments and their users
• the allocation of budgets to departments and users
• the definition of the period of time the allocated budget is valid for (day, week, month, year, concrete period)
• the definition of the frequency of the period of time (1 / 2 / 3 etc. days / weeks / months/ etc.) and the start date
• the logging of the budget consumption per department and users
• the blocking of the generation of content when the budget for the user or the department the user belongs to is zero