10.5 Firefly Generative AI, Generative Fill, and Neural Filters

Firefly Generative AI, Generative Fill, and Neural Filters

Quick Answer: Adobe Photoshop incorporates cutting-edge generative artificial intelligence and machine learning models powered by Adobe Firefly and Adobe Sensei. Generative Fill operates via the Contextual Task Bar, utilizing an active pixel selection and descriptive text prompt to generate photorealistic imagery—leaving the prompt blank performs seamless, context-aware object removal. Content is generated on an independent Generative Fill Layer with attached masks and multi-variation state management in the Properties panel. Generative Expand pairs with the Crop tool (C) to extend canvas boundaries seamlessly. Complementing generative synthesis, Sky Replacement automates atmospheric relighting, and the Neural Filters workspace executes deep learning tasks including Skin Smoothing, Smart Portrait, Colorize, and Photo Restoration.

---

1. Adobe Firefly Generative AI: Architecture and Commercial Safety

Adobe Firefly is Adobe's proprietary family of creative generative AI foundation models, integrated natively into the core Photoshop document engine.

Commercial-use terms and output review

Adobe describes Firefly’s training sources as licensed content, including Adobe Stock, and public-domain material. This does not guarantee that every output is free of third-party rights or that every customer receives indemnification. Check the selected model’s terms, client policy, and depicted brands or people before distribution. Photoshop can offer different models or services in later releases; do not apply one model’s terms to all of them.

Processing Pipeline: Cloud Architecture and Generative Credits

• Cloud Synthesis: Generative Fill does not compute locally on the user's CPU or GPU. Prompts, selection boundaries, and surrounding context pixels are encrypted and transmitted to Adobe's cloud computing infrastructure, where high-performance GPU clusters synthesize the imagery.
• Connectivity Requirement: Generating or regenerating variations requires an active internet connection.
• Generative Credits: Monthly allocations of Creative Cloud generative credits meter the generation rate, resetting on every billing cycle.

---

2. Generative Fill: Workflow, Architecture, and Variations

Generative Fill enables creators to add, replace, or completely eliminate visual elements using natural language descriptions.

[Generative Fill Workflow in Photoshop]

 1. Active Selection ───► 2. Contextual Task Bar ───► 3. Generative Fill Layer Generated
    ┌─────────────┐        ┌─────────────────────────┐   ┌───────────────────────────────┐
    │  (Lasso /   │        │ [Generative Fill]       │   │ Layers Panel:                 │
    │   Marquee / │ ────►  │ ["vintage leather bag"] │ ─►│  [★] Generative Fill 1        │
    │   Obj Sel)  │        │ [ Generate ]            │   │  [Mask] Attached Mask         │
    └─────────────┘        └─────────────────────────┘   └───────────────────────────────┘
                                                                         │
                                                                         ▼
                                                         [Properties Panel Variations]
                                                          ┌───────┬───────┬───────┐
                                                          │ Var 1 │ Var 2 │ Var 3 │
                                                          └───────┴───────┴───────┘

The Three Primary Workflows:

1. Adding New Objects to a Scene

1. Select any selection tool (Lasso L, Rectangular Marquee M, or Object Selection W) and draw a selection where the object should appear.
2. In the floating Contextual Task Bar, click the Generative Fill button.
3. Type a descriptive, concise prompt in natural language (e.g., "yellow vintage sports car parked on asphalt") and click Generate.
4. Firefly matches the lighting angle, shadows, perspective convergence, depth of field (lens bokeh), and color temperature of the background scene automatically.

2. Seamless Object Removal (The Blank Prompt Technique)

To remove an unwanted object (such as a tourist, trash receptacle, power line, or photobombing car):

1. Draw a loose selection around the object with the Lasso tool, ensuring you include a small boundary of surrounding context pixels.
2. Click Generative Fill on the Contextual Task Bar.
3. Leave the text prompt completely blank.
4. Click Generate.

3. Replacing Backgrounds

1. Use Select > Subject to isolate foreground people or products.
2. Invert the selection via Select > Inverse (Cmd+Shift+I / Ctrl+Shift+I).
3. Click Generative Fill, enter a prompt (e.g., "cozy minimalist Scandinavian living room with soft window lighting"), and click Generate.

The Generative Fill Layer Architecture

Executing Generative Fill creates a specialized Generative Fill Layer in the Layers panel:

• Unique Layer Badge: Displays a distinctive generative spark icon (★).
• Non-Destructive Mask: Automatically incorporates a black-and-white layer mask corresponding to the initial active selection area.
• Zero Pixel Destruction: Original background pixels beneath remain completely unaltered.

Managing Variations in the Properties Panel

Every time Generate is clicked, Firefly synthesizes three distinct visual variations displayed as thumbnails in the Properties panel:

• Clicking a variation thumbnail instantly swaps the canvas preview.
• If none of the initial variations are ideal, click the Generate button in the Properties panel again to create three additional variations without discarding previous results.
• Unwanted variations can be deleted by clicking the trash can icon on the thumbnail.
• Variations remain stored inside the .psd or .psdc file, ensuring client revisions can swap options instantly days or weeks later.

---

3. Generative Expand: Aspect Ratio and Canvas Un-Cropping

Generative Expand pairs generative synthesis with the Crop Tool (C), enabling designers to expand canvas boundaries beyond original photographic limits without distortion or stretching.

[Original 1:1 Square Photo]      [Generative Expand to 16:9 Landscape]
     ┌─────────────┐              ┌─────────┬─────────────┬─────────┐
     │             │              │ (AI Gen │  Original   │ (AI Gen │
     │   Subject   │    ───►      │  Left)  │   Subject   │  Right) │
     │             │              │         │             │         │
     └─────────────┘              └─────────┴─────────────┴─────────┘
                                 (Seamlessly synthesizes extended horizon,
                                  trees, and lighting matching original lens)

Production Execution:

1. Select the Crop Tool (C).
2. On the Options bar, locate the Fill dropdown menu and change it from Default or Content-Aware to Generative Expand.
3. Drag the crop handles outward past the image edges (or enter a target aspect ratio, such as 16 : 9 or 9 : 16, in the ratio fields).
4. In the Contextual Task Bar, either leave the prompt blank (to let AI naturally extrapolate the scene) or enter a descriptive prompt to direct what appears in the new margins.
5. Click the checkmark (✓) or press Enter. Photoshop generates the new canvas area on a dedicated Generative Fill layer with three selectable variations.

---

4. Sky Replacement: AI Horizon Detection and Atmospheric Relighting

Accessed via Edit > Sky Replacement, this feature leverages Adobe Sensei machine learning algorithms to identify, mask, and replace overcast or bland skies with vibrant atmospheric alternatives.

[Sky Replacement Control Architecture]
 ┌───────────────────────────┬─────────────────────────────────────────────────┐
 │ Sky Preset Selector       │ Sky Adjustments                                 │
 │  ▶ Blue Skies             │   Shift Edge:   [ -10 px ]                      │
 │  ▼ Sunsets                │   Fade Edge:    [ +25 px ]                      │
 │    [■ Sunset_01.jpg]      │   Brightness:   [ +10 ]                         │
 │    [■ Sunset_02.jpg]      │   Temperature:  [ +15 ]  (Cool ◄──► Warm)       │
 │  ▶ Spectacular            │   Scale:        [ 105% ]                        │
 │  [+] Custom Sky Import    │   Flip:         [✓] Flip Horizontal             │
 ├───────────────────────────┼─────────────────────────────────────────────────┤
 │ Output Destination        │ Foreground Adjustments                          │
 │ (•) New Layers            │   Lighting Mode:       [ Multiply ▼ ]           │
 │ ( ) Duplicate Layer       │   Lighting Adjustment: [ +20 ]                   │
 │                           │   Color Adjustment:    [ +45 ] (Harmonizes Tint)│
 └───────────────────────────┴─────────────────────────────────────────────────┘

Sky Replacement Controls Explained:

1. Sky Edge Refinement

• Shift Edge: Moves the boundary between the sky and foreground elements inward or outward. Essential for preventing white halos or clipping around complex silhouettes like tree branches and telephone poles.
• Fade Edge: Adjusts the feathering softness of the transition gradient along the horizon.

2. Sky Adjustments

• Brightness & Temperature: Calibrates the luminance and warmth/coolness of the replacement sky.
• Scale & Flip: Scales the sky graphic to fit the composition; checking Flip horizontally mirrors the sky so that cloud illumination matches the natural shadow direction of the foreground scene.

3. Foreground Lighting and Color Harmonization

Replacing a bright daytime sky with a dramatic crimson sunset produces an obvious fake composite if the foreground remains lit by daylight. Sky Replacement solves this via Foreground Adjustments:

• Lighting Mode: Toggles between Multiply (darkens foreground to match twilight/night skies) and Screen (brightens foreground).
• Lighting Adjustment: Scales foreground exposure to match new ambient sky light.
• Color Adjustment: Samples the dominant chromatic colors of the new sky and dynamically tints foreground objects, harmonizing the entire composition.

---

5. The Neural Filters Workspace: Deep Learning Solutions

Accessed via Filter > Neural Filters, this dedicated workspace harnesses deep convolutional neural networks powered by Adobe Sensei to perform complex photo retouching tasks in seconds.

[Neural Filters Workspace Organization]
 ┌─────────────────────────┬─────────────────────────────────────────────────────┐
 │ Filter Catalog          │ Active Filter Parameters                            │
 │ ── Featured ──          │                                                     │
 │ [✓] Skin Smoothing      │   Skin Smoothing:                                   │
 │ [✓] Smart Portrait      │     Blur:      [ 45 ]  Smoothness: [ 30 ]           │
 │ [ ] Style Transfer      │                                                     │
 │ [✓] Colorize            │   Smart Portrait:                                   │
 │ [ ] Depth Blur          │     Happiness: [ +15 ]                              │
 │ ── Beta Filters ──      │     Facial Age:[ -10 ]                              │
 │ [✓] Photo Restoration   │     Hair:      [ +25 ] (Thickens hair)              │
 │ [✓] JPEG Artifacts      │     Gaze/Head: [ -5° ]                              │
 │                         │     Light Dir: [ +30°] (Re-casts shadow)            │
 └─────────────────────────┴─────────────────────────────────────────────────────┘

Cloud vs. Local Filter Execution

• Local Execution: Optimized filters (such as Skin Smoothing and JPEG Artifacts Removal) run directly on the local workstation using hardware acceleration (GPU / Apple Neural Engine).
• Cloud Execution: High-compute models (such as Smart Portrait and complex Style Transfer) offload processing to Adobe cloud AI servers.
• Downloading Filters: Filters marked with a small cloud/arrow icon require a one-time download to the local machine before first use.

Core Neural Filters for Professional Workflows:

1. Skin Smoothing

Automatically maps human facial topography, selectively smoothing skin blemishes, acne, and surface roughness while preserving tack-sharp focus on eyelashes, eyebrows, lips, and pores.

2. Smart Portrait

Manipulates expressive facial parameters using deep learning synthesis:

• Happiness, Surprise, Anger: Shifts emotional expression naturally without manual warping.
• Facial Age: Adjusts wrinkles, skin elasticity, and hair color to make subjects appear younger or older.
• Hair Thickness: Dynamically synthesizes hair volume.
• Head Direction & Eye Direction: Reorients facial angles in 3D space.
• Light Direction: Re-casts facial highlights and shadows as though a virtual strobe moved across the set.

3. Colorize

Automatically analyzes historical black-and-white photographs and estimates plausible colors rather than recovering historically verified colors for skin, clothing, vegetation, and architecture using deep learning trained on millions of color images. Designers can click on the preview image to drop manual color pins, dictating specific hues for historic military uniforms or corporate branding.

4. Photo Restoration

Designed for vintage, damaged physical prints. Leverages machine learning to eliminate physical cracks, deep scratches, dust spots, and paper grain while restoring contrast and facial sharpness.

5. JPEG Artifacts Removal

Cleans up blocky 8×8 DCT compression artifacts, high-frequency ringing, and color smearing from heavily compressed web images.

Output Destination Options in Neural Filters:

Always evaluate the Output dropdown at the bottom of the Neural Filters panel:

• Current Layer: Destructive; overwrites active pixel data.
• New Layer: Outputs processed pixels onto an independent layer.
• New Layer Masked: Generates a new layer with an attached transparency mask.
• Smart Filter (Recommended): If the layer was converted to a Smart Object, outputs as a fully editable, non-destructive Smart Filter.
• New Document: Duplicates the composition into a new document file.

---

6. Exam Scenarios and Common Pitfalls

Scenario 1: Retoucher Over-Types Prompts for Object Removal

A retoucher needs to remove a fire hydrant from a brick sidewalk. In the Contextual Task Bar, they type "remove fire hydrant, replace with brick sidewalk, seamless pattern, high quality". The resulting generation places an odd metal grate or a miniature flower pot instead.

• The Root Cause: Firefly interprets descriptive text as an instruction to add content. Words like "remove" or "replace" confuse the semantic synthesis engine.
• The Professional Fix: Select the fire hydrant and leave the prompt completely blank. Firefly recognizes the absence of text as an instruction for contextual background synthesis, seamlessly reconstructing the brick sidewalk.

Scenario 2: Sky Replacement Produces Obvious "Cutout" Composite

A designer inserts a sunset sky behind a building photographed at noon. While the sky looks beautiful, the building still has bright cool-blue midday reflections and harsh downward shadows, looking completely disconnected from the background.

• The Root Cause: The designer overlooked the Foreground Adjustments settings inside the Sky Replacement dialog.
• The Professional Fix: In the Sky Replacement dialog, adjust Lighting Mode to Multiply, increase the Lighting Adjustment slider to darken the facade, and advance the Color Adjustment slider to transfer the warm sunset amber tint onto the building's walls.

Summary Checklist for Generative AI and Neural Tools

1. For object removal with Generative Fill, did you remember to leave the text prompt completely blank?
2. Did you inspect all three generated variations in the Properties panel before committing?
3. When expanding canvas boundaries with the Crop tool, did you switch the Fill dropdown to Generative Expand?
4. In Sky Replacement, did you configure Color Adjustment to harmonize foreground lighting with the new sky, and output to New Layers?
5. In Neural Filters, did you set the Output mode to Smart Filter or New Layer to protect original pixels?

Exam-version boundary

The current 2025 objective 5.4 explicitly excludes Neural Filters from the assessed filter topic. The Neural Filters discussion here is optional workflow context. Prioritize raster painting, selections, Remove, adjustment layers, standard filters, layer styles, and materials for this objective set. Internet access, account eligibility, model availability, and credit rules can change; confirm the test center’s supported exam version rather than installing an arbitrary newest release. Source: current Photoshop objectives, checked October 18, 2026.