o i_@sddlZddlZddlZddlmmZddlZddl Zddl Z ddl m Z ddlmZmZddlmZddlmZmZdZdded ejfd ejd ed ed ejdejdejf ddZ  d;dejdejdeededededefddZ dN)override)ComfyExtensionio)parse_json_tracks)Image ImageDrawFc Cs$tdtdt|tj|S)Ni')torchpowmuldivtofloat32)idr8/mnt/c/Users/fbmor/ComfyUI/comfy_extras/nodes_wanmove.py$rcpupos_k pos_emb_dim theta_funcdevicedtypereturnc Cs|ddks Jd|||}tr|d}|d}tjd|d||d}|dd|d}|||}|dd|} tt| |} t t| |} tj | | gdd} | S)Nrrz2The dimension of position embeddings must be even.rrdim) r SKIP_ZEROsizer arangeviewexpandsinr coscat) rrrrr batch_size denominatordenominator_expandedthetaspos_k_expanded sin_thetas cos_thetaspos_embrrr get_pos_embs   r1rsample pred_trackspred_visibilitydownsample_ratiosheightwidth track_numt_down_strategycCs|dvsJd|j\}}} |\} } } tj||d| ddtjd } |dkr*|}t|d|}|dd|f}|dd|f}td|| D]~}|dksS|dkr\||}||}n|||| jdd }tj|||| dd }t|D]M}||r||ddks||ddks||d|ks||d|krqy||\}}t|| t|| }}||| ||| df<| ||| df<qyqI| S) N)r2averagez1Invalid strategy for downsampling time dimension.rr)rrrr2r) shaper oneslongrandpermrangemeananyint)r3r4r5r6r7r8r9tn_t_downh_downw_down track_pos tracks_idxtracks visibilityt_idx cur_trackscur_visibilityrxyrrrcreate_pos_embeddings/s.  "  H (rR? vae_featurerIstrengthcCs|j\}}}}}||jdksJd|jd}|ddt|ddddf}|ddddddddf} | ddk| ddk@} | jdd} | jd} | dkrV|S| dddf} | dddf}| dddf}|d}| | ||df}| | ||df}|| |ddf}|| |ddf}|| ddd||f}|| dd|||f}|||||| dd|||f<|S) NrzBatch size mismatch.r).r).rF)as_tupler)r;r r>nonzeror=)rTrIrUbrErChwrD current_posmask valid_indices num_valid batch_idx track_idxt_relt_targeth_targetw_targeth_sourcew_source src_features dst_featuresrrrreplace_featureVs, "   "ricCst|d}|ddd}d}g}tt|dD]1}||dd||d} ||dd||d} | | | | d} || || 7}q|dkrQdSd} tt|dd|ddD]x\} \}}|| }tt|d}t|D]^}| |||}||}tdd||}g||R}t|d|d|d||}t|d|d|d||}tt|d|d}|j ||f|d|fg||dqw| |7} qbdS)NRGBArrr?)fillr7) rDrawr?lenappend enumeratezipmaxrBline)overlay line_widthpoints start_coloropacitydraw total_lengthsegment_lengthsrdxdylengthaccumulated_lengthidx start_point end_pointsegment_lengthstepscurrent_lengthratioalphacolorrPrQdynamic_line_widthrrr"_draw_gradient_polyline_on_overlays6   * $$" rcCs~t|}t|}|dddddfd}tj|gddd}|ddddddf||d|}t|tjS)No@r)axisr)nparraystackr fromarrayastypeuint8)rgbtrackr blend_imgrrr add_weighteds  *r rkcCsjgd}|}|d}|dur&|d}|jdd\}} } |jd} td|} g} t|D]}||t j }t d| | fd}t |}g}t| D]Y}|durn|||fdkrnq_|||f}||t|}|| f}|j|d||d||d||d|f|d|t||d|d|f}t|dkr|||fq_t |}|dddddd fd }|ddddddf||d|}|r&t d| | fd}|D] \}}t|||||qt |}|dddddd fd }|ddddddf||d|}| t |t jqA| S) N))frl)rrlrl)rlrlr)rlr)rrlrrrrrlrj)rrrr)rmr)byternumpyr=detachr;rBr?rrrrnewrrnroellipsersrprrrr)videorKrL track_frame circle_sizeryrv color_map num_framesr6r7 num_tracks alpha_opacity output_framesrC frame_rgbru draw_overlay polyline_datarD track_coordr circle_color tracks_coord overlay_nprpolyline_overlay polyline_nprrrdraw_tracks_on_videosN       .  *  *rc@.eZdZeddZeddejfddZdS)WanMoveVisualizeTrackscCstjddtjdtjjdddtjjddd d d tjjd d d dddtjjddddddtjjddd dddgtjgdS)Nrconditioning/video_modelsimagesrKToptionalline_resolutionrrdefaultminrsrr)rrrsadvancedryg?rS{Gz?rrrssteprvrnode_idcategoryinputsoutputs)rSchemarInputTracksIntFloatOutputclsrrr define_schemas  z$WanMoveVisualizeTracks.define_schemaNrc Cs|dur t|S|dd}|dd}|d} | jd|jdkr7|jd|jd} | | ddd} t| ||||||d} tjdd| Ddd dd  } t| t j S) N track_pathrtrack_visibilityrr)rrryrvcSsg|]}t|qSr)TF to_tensor).0framerrr sz2WanMoveVisualizeTracks.execute..rr)r NodeOutput unsqueezer;repeatrr rmovedimfloatr comfymodel_managementintermediate_device) rrrrryrvrKrr images_in repeat_count track_videorrrexecutes $zWanMoveVisualizeTracks.executeN__name__ __module__ __qualname__ classmethodrrrrrrrrrs  rc@r)WanMoveTracksFromCoordsc CsFtjddtjjdddddtjjdddgtjtjjd d gd S) Nrr track_coordsTz[]) force_inputrr track_maskr track_length display_namer)rrStringrMaskrrrrrrrrs z%WanMoveTracksFromCoords.define_schemaNrc stj}t|td}fddttdD}tj|tj|d}|j d}|durWanMoveTracksFromCoords.execute...r)r tracks_datarrr0sz3WanMoveTracksFromCoords.execute..rrrrrrrr)rrrrror?r tensorrr;r<boolrArrr) rrrrr track_listrKrrout_track_inforrrr)s     zWanMoveTracksFromCoords.executerrrrrrrs  rc@s0eZdZeddZe d dejfddZdS) GenerateTrackscCsFtjdgddtjjddddddtjjd d ddddtjjd d d d dddtjjdd d d dddtjjdd d d dddtjjdd d d dddtjjdddddtjjdddddtjjdd d d d!d"dtjjd#d$d%d&tjjd'd(d d dd)dtjjd*d(d d dd+dtjjd,gd-d.d/tjjd0d1d2d3gtj tjj d4d5gd6S)7Nr)z motion pathszcamera movement trajectoryrr7@rrr6start_xrrSrz1Normalized X coordinate (0-1) for start position.rrrsrtooltipstart_yz1Normalized Y coordinate (0-1) for start position.end_xz/Normalized X coordinate (0-1) for end position.end_yz/Normalized Y coordinate (0-1) for end position.rQrrrrd track_spreadg?gMbP?z\Normalized distance between tracks. Tracks are spread perpendicular to the motion direction.bezierFz>Enable Bezier curve path using the mid point as control point.)rr mid_xrkzPNormalized X control point for Bezier curve. Only used when 'bezier' is enabled.mid_yzPNormalized Y control point for Bezier curve. Only used when 'bezier' is enabled. interpolation)linearease_inease_out ease_in_outconstantz5Controls the timing/speed of movement along the path.)optionsr rTz)Optional mask to indicate visible frames.)rr rr)rsearch_aliasesrrr) rrrrrBooleanComborrrrrrrrCs6 zGenerateTracks.define_schemaFrNrc/Cstj}| }||}||}||}||}||}||}| ||d}tjdd| |d}| dkr:t|}n+| dkrA|}n$| dkrJ|d}n| dkrWdd|d}n| d kre||d d|}| r|}d|}|d|d||||d|}|d|d||||d|}d|||d|||}d|||d|||}n ||||}||||}t|||}t|||}g} t| D]_}!||!}"||!}#|"d|#dd }$|$dkr|# |$}%|"|$}&nd }%d }&g}'t| D](}(|(| dd|})||!|%|)}*||!|&|)}+|' |*|+gq | |'qtj | tj |d},|durVtj || ftj |d}-n |dkjddd}-i}.|,|.d<|-|.d<t|.|S)NrrrrrrrrrrrkrSrrrrrrr)rrrr linspace zeros_like full_liker?itemrprrr<rrArrr)/rr7r6rr rrr r rrrrrrrr start_x_px start_y_pxmid_x_pxmid_y_pxend_x_pxend_y_pxtrack_spread_pxrC interp_valuest_interp one_minus_t x_positions y_positions tangent_x tangent_yr frame_idxtxtyrperp_xperp_y frame_tracksr`offsettrack_xtrack_yrKrrrrrrcsp   (( "          zGenerateTracks.execute)FrNrrrrrrBs rc@r)WanMoveConcatTrackcCs2tjddtjdtjjdddgtjgdS)Nr:rtracks_1tracks_2Trr)rrrrrrrrrrs z WanMoveConcatTrack.define_schemaNrcCsd|dur t|Stj|d|dgdd}tj|d|dgdd}i}||d<||d<t|S)Nrrrrr)rrr r()rr;r< tracks_outmask_outrrrrrs  zWanMoveConcatTrack.execute)NNrrrrrr:s   r:c@r)WanMoveTrackToVideocCstjddtjdtjdtjdtjjdddtjjd d d d d ddtjjdddtj ddtjjdddtj ddtjjdddtj ddtjjdddddtj dtj jdddg tjj ddtjj ddtj j d dgd!S)"Nr?rpositivenegativevaerKTrrUrSrgY@rz#Strength of the track conditioning.rr7rrrr6rrr rrr)rr start_imageclip_vision_outputrlatentr)rr ConditioningrVaerrrnodesMAX_RESOLUTIONrClipVisionOutputrLatentrrrrrs(       z!WanMoveTrackToVideo.define_schemaNrc Cstj} tj|d|ddd|d|dg| d} | durtj| d|dd||dddd} tj|||| j df| j | j d d }| |d| j d <| |dddddddd f}tjdd| j d |j d|j df| j | j d }d|ddddd| j d dddf<| dur|dkr| dd|}|j d}| dtj||ftj| d}t||gd|||d}tj|d |}t|||}n|}t|||d}t|||d}| durt|d| i}t|d| i}i}| |d<t|||S)Nrrrrrbilinearcenterrrkrrrrrrrr)rrLrL)r8)concat_latent_image concat_maskrDsamples)rrrr zerosutilscommon_upscalerr<r;rrencodegetrrRresize_to_batch_sizerri node_helpersconditioning_set_valuesrr)rr@rArBr7r6rr)rUrKrCrDrrEimagerOr\ tracks_pathrrrIconcat_latent_image_pos out_latentrrrrs2 ,,&&0, zWanMoveTrackToVideo.execute)NNNrrrrrr?s  r?c@s(eZdZedeeejfddZdS)WanMoveExtensionrcstttttgSr)r?rr:rr)selfrrr get_node_list szWanMoveExtension.get_node_listN) rrrrlisttyper ComfyNoder`rrrrr^ sr^cstSr)r^rrrrcomfy_entrypointsrd)rr2)rS)Nrrrkr).rHrXr !torchvision.transforms.functional transforms functionalrcomfy.model_managementr comfy.utilsrrtyping_extensionsrcomfy_api.latestrrcomfy_extras.nodes_wanrPILrrr!rrTensorrBcallablerr1rastrrRrrirrrrcrrrr:r?r^rdrrrrs   $ * 0( =$)n?