IC-Light

IC-Light (Imposing Consistent Light) is a cutting-edge project designed to manipulate and harmonize illumination in images using deep learning techniques. The project currently offers two types of models: a text-conditioned relighting model and a background-conditioned model. Both models take foreground images as inputs and allow users to modify lighting conditions based on textual descriptions or background references.

Key Features:

• Text-Conditioned Relighting: Users can specify lighting preferences (e.g., "Left," "Right," "Bottom") and textual prompts (e.g., "sunshine from window," "neon light") to achieve desired lighting effects.
• Background-Conditioned Relighting: This model harmonizes the lighting of the foreground with a given background, ensuring visual consistency.
• High Consistency: The models are trained to impose consistent light transport, enabling highly realistic relighting effects that can even be merged into normal maps.

Technical Highlights:

• Built on latent diffusion models (LDM) with specialized MLPs to enforce lighting consistency.
• Supports both HDR and standard lighting adjustments.
• Includes pre-trained models like iclight_sd15_fc.safetensors (text-conditioned) and iclight_sd15_fbc.safetensors (background-conditioned).

Use Cases:

• Portrait Relighting: Adjust lighting for portraits to match different environments (e.g., indoor, outdoor, studio).
• Creative Effects: Generate stylized lighting (e.g., neon, sunset, cyberpunk) for artistic projects.
• Background Replacement: Harmonize foreground lighting with new backgrounds for seamless compositing.

IC-Light is ideal for photographers, graphic designers, and developers looking to automate or enhance image lighting workflows. The project is open-source, with models available for download via GitHub, and includes a Gradio demo for easy experimentation.