Try to find the beginning of your story (not always easy to do), and start Chapter 1 there; this may or may not be the same as the inciting incident. If the transitions are handled well enough, the jumps back and forth shouldn't be jarring - so the prologue might not be necessary at all. Maybe have some beta readers explain their disconnect. Pro tip: always listen to readers when there is a problem, but never take their advice on how to fix it. I think that one comes from Orson Scott Card, but don't quote me.
The first thing to know is that you're writing a portal fantasy. You have the familiar world on this side of the portal, where all of the science (or magic) stuff happens that creates or reveals the portal, in your case time travel; and then you have the through-the-looking-glass portal world, again, in your case, the 1600s.
In a story like Outlander (magic time travel), the story bounces around all over the place in pursuit of character and establishing conflicting relationships. In a story like Crichton's Timeline, another time travel adventure, we bounce around way less. Once there, we stay mainly in the past, with the exception of showing the unstable tech in the 'present' that might prevent the team from returning, thus amping up the suspense.
Outlander is a deeper story and Timeline is a more superficial read, neither is objectively better or worse, they're just different approaches to the same setup, each focusing on different narrative elements.
A common approach that avoids the whole opening discussion is In medias res - the currently preferred adaptation being opening up mid-action. Shallow readers, meaning those that prefer to stay on the surface of things for a lighter read, usually enjoy these openings - Bam! Bam! Zoom! Zoom! Pow! But the deeper reader typically hates them because without establishing character and motivation, the action falls flat. It doesn't matter.
So, depending on your story, one option is to start mid-action in the 1600s (which would be the look inside), end on a cliffhanger, and then return to the 'present' to explain the whole time traveling thing; from what you're saying, that might mean moving the two prologue chapters to Chapters 2 and 3. Again, there’s no right or wrong, just different approaches.
With that said, I think books need to establish character as soon as possible. We need someone to root for. Cussler writes wonderful and fairly lengthy prologues that establishes a mystery in the past that then intertwines with the present-day plot. He doesn’t have to worry about character because his stories always feature the same guy - Dirk Pitt. Cussler is really good with mysteries but he's a pretty superficial writer, that is, he tends to stay on the surface of things (no water pun intended). Good and evil are usually black and white; Cussler doesn’t trade in moral ambiguity.
So, if the present day stuff gets to character first, start there. If the character stuff tends to come during the adventures of the past, start there. You can also weave the present day familiar world science stuff in as you go, with either flashbacks, discussions or memories - or just time jumps. The cool thing about time travel adventures is time is relative. While I think it's important to maintain a reasonably linear narrative, so as not to confuse readers, you can play around with it a little bit at the edges.
Good luck.