The actor-director, 40, is no longer a client of WME as of the morning of Saturday, Dec. 21, according to multiple outlets including Deadline, which was first to report the news.
The decision to drop Baldoni was made in part due to the complaint filed by his It Ends with Us costar Lively, according to Deadline. The complaint accuses Baldoni of sexual harassment, among other claims. A client of WME herself, Lively, 37, continues to be represented by the agency.
Related: Everything Blake Lively Claims in Complaint Against Justin Baldoni: Unwanted Kissing, Watching Her Naked and Smear Campaign
WME did not comment when reached by PEOPLE, and a representative for Baldoni did not immediately respond to PEOPLE's request for comment.
News of Baldoni's departure from WME comes on the heels of Lively's complaint -- a precursor to filing a discrimination lawsuit in California -- accusing Baldoni of "invasive, unwelcome, unprofessional and sexually inappropriate behavior" on the set of It Ends with Us.
Never miss a story -- sign up for PEOPLE's free daily newsletter to stay up to date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
In the complaint, which comes months after rumors of a feud between the costars swirled, the actress outlines dozens of allegations against Baldoni (who starred in, produced and directed the film) as well as other members of the production team and the individuals said to be involved in the alleged smear campaign against her.
The complaint alleges that Baldoni "improvised" kisses and other physically intimate moments in the movie, added "gratuitous sexual content" and made numerous inappropriate comments that were sexual in nature.
According to the complaint, the director also allegedly displayed a "shocking lack of boundaries" by entering the actress' trailer "uninvited" while she was undressed or "vulnerable" -- "including when she was breastfeeding her infant child."
The complaint also alleges that ahead of It Ends with Us' August premiere, Baldoni, fearing that Lively would go public with her negative on-set experiences, enlisted a team to "bolster" his credibility and "eviscerate" his costar's reputation.
Related: It Ends with Us Cast Drama Explained: What Is Going on with Blake Lively and Justin Baldoni?
Bryan Freedman, Baldoni's lawyer, told PEOPLE in a statement that Lively's complaint was to "fix her negative reputation." He also said that her allegations are "false, outrageous and intentionally salacious with an intent to publicly hurt."
Freedman also alleged that Lively caused issues during the filming of the movie, including "threatening to not show up to set, threatening to not promote the film, ultimately leading to its demise during release."
Lively, meanwhile, issued a statement to The New York Times, saying, "I hope that my legal action helps pull back the curtain on these sinister retaliatory tactics to harm people who speak up about misconduct and helps protect others who may be targeted."