The station was evacuated, reports The Associated Press, train service suspended and police warned people to stay away.
"An odious attack," tweeted French Interior Minister Gérard Collomb. "A cowardly and abject crime."
After having viewed video footage of the incident, Collomb said the suspect attacked one woman before fleeing, then returned to kill the second victim, reports Le Monde. Collomb called both victims "young women" and added that the quick-acting soldiers possibly saved others from becoming victims of the attack.
"This act could be terrorist in nature, but at this time we cannot confirm it," Collomb said at a press briefing in front of the station.
France has been the site of numerous terror attacks in recent years; the worst of which unfolded on the night of Nov. 13, 2015 when gunfire and grenades exploded across Paris, including at the Bataclan concert hall. In all, 130 people died in the attack claimed by the Islamic State.
The soldiers on patrol Sunday in Marseille were working as part of Opération Sentinelle, which has seen thousands of combat troops deployed to guard transportation hubs, religious centers and tourist attractions across the country.
The operation was launched in January 2015, after Islamic extremists opened fire at the Paris-based satirical newspaper Charlie Hebdo and a Kosher grocery store, killing 17 people.
Since 2015, more than 230 people across France have been killed in terrorist attacks, reporter Jake Cigainero says from Paris.
On Sunday, French President Emmanuel Macron expressed "deep indignation for the barbarous act," and saluted the police and soldiers "who reacted with sang-froid and efficiency."
Earlier this month, four American university students studying abroad were splashed with acid at Saint Charles, the same train station that was the scene of Sunday's attack. But authorities said the suspect in that case was suffering from a mental illness and did not consider it an act of terror.