DEV.co
Open-Source Databases · yitter

IdGenerator

IdGenerator is a multi-language distributed ID generation library implementing an optimized Snowflake algorithm variant called "Snowflake Drift." It generates shorter numeric IDs with very high throughput (50W/0.1s claimed) and supports automatic WorkerId registration in containerized environments.

Source: GitHub — github.com/yitter/IdGenerator
3.2k
GitHub stars
422
Forks
C
Primary language
MIT
License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

FieldValue
Repositoryyitter/IdGenerator
Owneryitter
Primary languageC
LicenseMIT — OSI-approved
Stars3.2k
Forks422
Open issues35
Latest releasev1.3.3 (2023-06-20)
Last updated2025-04-10
Sourcehttps://github.com/yitter/IdGenerator

What IdGenerator is

A Snowflake algorithm implementation with optimizations for ID length reduction and concurrent throughput. Provides native support for C#, Java, Go, Rust, C, JavaScript, TypeScript, and Python, with configurable bit lengths for timestamp, WorkerId, and sequence components. Handles clock skew via time-drift buffering and supports manual ID insertion via reserved bits.

Quickstart

Get the IdGenerator source

Clone the repository and explore it locally.

terminalbash
git clone https://github.com/yitter/IdGenerator.gitcd IdGenerator# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Distributed database primary keys

When you need globally unique, monotonic-increasing numeric keys across multiple sharded databases with shorter storage footprint than traditional Snowflake or GUID, and need to reflect business time ordering.

High-concurrency ID generation with container auto-scaling

Kubernetes and container environments requiring automatic WorkerId registration (via Redis) to maintain uniqueness across horizontally scaled service instances without external ID service dependency.

JavaScript-compatible numeric IDs

Applications needing IDs that fit within JavaScript Number.MAX_SAFE_INTEGER (9007199254740992) to avoid string conversion overhead, especially when IDs are frequently transferred across frontend/backend boundaries.

Implementation considerations

  • WorkerId must be globally unique across all instances; failure to enforce this will cause ID collisions. In container environments, use the provided Redis-based auto-registration or build equivalent coordination.
  • Configure SeqBitLength and WorkerIdBitLength early; changing them after production use risks ID collisions. Increasing either is safer than decreasing, and their sum must not exceed 22 bits.
  • BaseTime (epoch) should be set once and never moved backward; if you must change it, only move it further into the past to maintain ID monotonicity.
  • Clock skew tolerance is built in but assumes small drifts (documented as "small-scale" time sync issues). Large manual clock resets may cause duplicate IDs if the system does not restart cleanly.
  • Treat core algorithm code as read-only; the library is mature (4+ years) but internal logic is complex and documented changes to config are extensive. Do not modify without rigorous testing.

When to avoid it — and what to weigh

  • You need guaranteed sequential IDs — The algorithm generates monotonic-increasing but not necessarily consecutive IDs. If your business requires fully sequential gaps-free numbering, this is not suitable.
  • You require built-in persistence or central coordination without Redis — Automatic WorkerId registration requires Redis. If you cannot operate Redis and need auto-scaling without manual WorkerId management, you must build a custom coordination layer.
  • System clock management is unreliable — Despite clock-skew handling, the algorithm depends on monotonic system time. Environments with frequent, uncontrolled clock adjustments or poor NTP synchronization will experience unpredictable behavior.
  • You need strong auditing of ID derivation — IDs encode time, WorkerId, and sequence in bit fields. Reverse-engineering the exact machine, datacenter, and timestamp requires knowing configuration parameters; this is not cryptographically opaque.

License & commercial use

Licensed under MIT (MIT License). Permissive OSI-approved license allowing commercial use, modification, and redistribution with minimal restrictions (attribution and license notice required).

MIT license permits commercial use and closed-source deployment. No license restrictions prohibit commercial applications. However, review the license file in the repository to confirm you include the MIT notice in your distribution.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

SignalAssessment
MaintenanceActive
DocumentationStrong
License clarityClear
Deployment complexityModerate
DEV.co fitStrong
Assessment confidenceHigh
Security considerations

The algorithm does not provide cryptographic properties; IDs are predictable if configuration parameters are known. Clock skew handling is operational (not cryptographic). No authentication/authorization layer exists; assume IDs are not secrets. Reverse-engineering machine/time from an ID is feasible; do not rely on ID opacity for security. Standard defenses against clock manipulation apply.

Alternatives to consider

Twitter Snowflake (original)

Standard reference implementation, widely used, lower maintenance. Generates longer IDs and lower throughput; no built-in clock-skew recovery or shorter ID optimizations. Suitable if you do not need ID compaction or extreme concurrency.

Ulid or Ksuid (UUID variants)

Sortable and collision-resistant. Larger (128-bit) keys, non-numeric; breaks JavaScript Number.MAX_SAFE_INTEGER. Better for immutable audit logs; worse for database indexing and numeric-only systems.

Central ID service (e.g., Leaf by Meituan, TinyId by Didi)

Centralized coordination removes distributed config complexity. Requires network roundtrip per ID batch; introduces single point of failure. Better for strongly consistent ID allocation; worse for latency-sensitive scenarios.

Software development agency

Build on IdGenerator with DEV.co software developers

IdGenerator is production-ready for high-concurrency systems needing numeric keys and container auto-scaling. Evaluate parameter tuning and Redis requirements for your scale, then implement via your language binding.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

Related on DEV.co

Explore the category and the services that help you build with it.

IdGenerator FAQ

Can I change parameters after deployment?
Increasing SeqBitLength or WorkerIdBitLength is safe. Decreasing them is dangerous and can cause collisions. BaseTime must never move backward. Consult the "Configuration Changes" section in the README before any parameter updates in production.
Do I need Redis?
No, unless you use automatic WorkerId registration in container/k8s environments. For static deployments, manually assign unique WorkerId values to each instance and skip Redis.
How do I handle clock skew?
The algorithm handles small clock drifts automatically via time-drift buffering and reserved sequence bits. Large manual clock resets (>1s backward) may cause collisions if the service does not restart; avoid manual clock adjustments and rely on NTP sync.
What is the throughput ceiling?
README claims 500M–3000M IDs/s on 8th-gen low-power i7. Practical throughput depends on SeqBitLength: 5W/s at default (SeqBitLength=6), up to 500W/s with SeqBitLength=12. Benchmark on your hardware and config.

Software development & web development with DEV.co

DEV.co helps companies turn open-source tools like IdGenerator into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source databases stack.

Ready to integrate distributed ID generation?

IdGenerator is production-ready for high-concurrency systems needing numeric keys and container auto-scaling. Evaluate parameter tuning and Redis requirements for your scale, then implement via your language binding.