-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Html to md feature #3099
Merged
Merged
Html to md feature #3099
Changes from 12 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
d97e62f
Import note from url with markdown
StormBurpee 4a9bc69
starting to write a test
StormBurpee 18aae8c
getting very close
StormBurpee 37eee26
fix linting & routing
AWolf81 44efb01
Merge branch 'master' into html-to-md
AWolf81 390f6d5
fix PropTypes
AWolf81 f67175e
fix test
AWolf81 558c091
fix linting
AWolf81 aeb77e5
Remove package-lock file & use startsWith for https check
AWolf81 a3f7d22
Add dracula theme styles
AWolf81 0d36f59
Create turndown service & use gfm turndown plugin
AWolf81 72b8d56
Merge remote-tracking branch 'upstream/master' into html-to-md
AWolf81 ec47ee8
Remove manual script tag filter and use turndown remove filter
AWolf81 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
const TurndownService = require('turndown') | ||
const { gfm } = require('turndown-plugin-gfm') | ||
|
||
export const createTurndownService = function () { | ||
const turndown = new TurndownService() | ||
turndown.use(gfm) | ||
return turndown | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
import PropTypes from 'prop-types' | ||
import React from 'react' | ||
import CSSModules from 'browser/lib/CSSModules' | ||
import styles from './FromUrlButton.styl' | ||
import _ from 'lodash' | ||
import i18n from 'browser/lib/i18n' | ||
|
||
class FromUrlButton extends React.Component { | ||
constructor (props) { | ||
super(props) | ||
|
||
this.state = { | ||
isActive: false | ||
} | ||
} | ||
|
||
handleMouseDown (e) { | ||
this.setState({ | ||
isActive: true | ||
}) | ||
} | ||
|
||
handleMouseUp (e) { | ||
this.setState({ | ||
isActive: false | ||
}) | ||
} | ||
|
||
handleMouseLeave (e) { | ||
this.setState({ | ||
isActive: false | ||
}) | ||
} | ||
|
||
render () { | ||
const { className } = this.props | ||
|
||
return ( | ||
<button className={_.isString(className) | ||
? 'FromUrlButton ' + className | ||
: 'FromUrlButton' | ||
} | ||
styleName={this.state.isActive || this.props.isActive | ||
? 'root--active' | ||
: 'root' | ||
} | ||
onMouseDown={(e) => this.handleMouseDown(e)} | ||
onMouseUp={(e) => this.handleMouseUp(e)} | ||
onMouseLeave={(e) => this.handleMouseLeave(e)} | ||
onClick={this.props.onClick}> | ||
<img styleName='icon' | ||
src={this.state.isActive || this.props.isActive | ||
? '../resources/icon/icon-external.svg' | ||
: '../resources/icon/icon-external.svg' | ||
} | ||
/> | ||
<span styleName='tooltip'>{i18n.__('Convert URL to Markdown')}</span> | ||
</button> | ||
) | ||
} | ||
} | ||
|
||
FromUrlButton.propTypes = { | ||
isActive: PropTypes.bool, | ||
onClick: PropTypes.func, | ||
className: PropTypes.string | ||
} | ||
|
||
export default CSSModules(FromUrlButton, styles) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
.root | ||
top 45px | ||
topBarButtonRight() | ||
&:hover | ||
transition 0.2s | ||
color alpha($ui-favorite-star-button-color, 0.6) | ||
&:hover .tooltip | ||
opacity 1 | ||
|
||
.tooltip | ||
tooltip() | ||
position absolute | ||
pointer-events none | ||
top 50px | ||
right 125px | ||
width 90px | ||
z-index 200 | ||
padding 5px | ||
line-height normal | ||
border-radius 2px | ||
opacity 0 | ||
transition 0.1s | ||
|
||
.root--active | ||
@extend .root | ||
transition 0.15s | ||
color $ui-favorite-star-button-color | ||
&:hover | ||
transition 0.2s | ||
color alpha($ui-favorite-star-button-color, 0.6) | ||
|
||
.icon | ||
transition transform 0.15s | ||
height 13px | ||
|
||
body[data-theme="dark"] | ||
.root | ||
topBarButtonDark() | ||
&:hover | ||
transition 0.2s | ||
color alpha($ui-favorite-star-button-color, 0.6) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
const http = require('http') | ||
const https = require('https') | ||
const { createTurndownService } = require('../../../lib/turndown') | ||
const createNote = require('./createNote') | ||
|
||
import { push } from 'connected-react-router' | ||
import ee from 'browser/main/lib/eventEmitter' | ||
|
||
function validateUrl (str) { | ||
if (/^(?:(?:(?:https?|ftp):)?\/\/)(?:\S+(?::\S*)?@)?(?:(?!(?:10|127)(?:\.\d{1,3}){3})(?!(?:169\.254|192\.168)(?:\.\d{1,3}){2})(?!172\.(?:1[6-9]|2\d|3[0-1])(?:\.\d{1,3}){2})(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])(?:\.(?:1?\d{1,2}|2[0-4]\d|25[0-5])){2}(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(?:(?:[a-z\u00a1-\uffff0-9]-*)*[a-z\u00a1-\uffff0-9]+)(?:\.(?:[a-z\u00a1-\uffff0-9]-*)*[a-z\u00a1-\uffff0-9]+)*(?:\.(?:[a-z\u00a1-\uffff]{2,})).?)(?::\d{2,5})?(?:[/?#]\S*)?$/i.test(str)) { | ||
return true | ||
} else { | ||
return false | ||
} | ||
} | ||
|
||
function createNoteFromUrl (url, storage, folder, dispatch = null, location = null) { | ||
return new Promise((resolve, reject) => { | ||
const td = createTurndownService() | ||
|
||
if (!validateUrl(url)) { | ||
reject({result: false, error: 'Please check your URL is in correct format. (Example, https://www.google.com)'}) | ||
} | ||
|
||
const request = url.startsWith('https') ? https : http | ||
|
||
const req = request.request(url, (res) => { | ||
let data = '' | ||
|
||
res.on('data', (chunk) => { | ||
data += chunk | ||
}) | ||
|
||
res.on('end', () => { | ||
const html = document.createElement('html') | ||
html.innerHTML = data | ||
|
||
const scripts = html.getElementsByTagName('script') | ||
for (let i = scripts.length - 1; i >= 0; i--) { | ||
scripts[i].parentNode.removeChild(scripts[i]) | ||
} | ||
|
||
const body = html.getElementsByTagName('body')[0].innerHTML | ||
const markdownHTML = td.turndown(body) | ||
|
||
html.innerHTML = '' | ||
|
||
if (dispatch !== null) { | ||
createNote(storage, { | ||
type: 'MARKDOWN_NOTE', | ||
folder: folder, | ||
title: '', | ||
content: markdownHTML | ||
}) | ||
.then((note) => { | ||
const noteHash = note.key | ||
dispatch({ | ||
type: 'UPDATE_NOTE', | ||
note: note | ||
}) | ||
dispatch(push({ | ||
pathname: location.pathname, | ||
query: {key: noteHash} | ||
})) | ||
ee.emit('list:jump', noteHash) | ||
ee.emit('detail:focus') | ||
resolve({result: true, error: null}) | ||
}) | ||
} else { | ||
createNote(storage, { | ||
type: 'MARKDOWN_NOTE', | ||
folder: folder, | ||
title: '', | ||
content: markdownHTML | ||
}).then((note) => { | ||
resolve({result: true, note, error: null}) | ||
}) | ||
} | ||
}) | ||
}) | ||
|
||
req.on('error', (e) => { | ||
console.error('error in parsing URL', e) | ||
reject({result: false, error: e}) | ||
}) | ||
req.end() | ||
}) | ||
} | ||
|
||
module.exports = createNoteFromUrl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
import PropTypes from 'prop-types' | ||
import React from 'react' | ||
import CSSModules from 'browser/lib/CSSModules' | ||
import styles from './CreateMarkdownFromURLModal.styl' | ||
import dataApi from 'browser/main/lib/dataApi' | ||
import ModalEscButton from 'browser/components/ModalEscButton' | ||
import i18n from 'browser/lib/i18n' | ||
|
||
class CreateMarkdownFromURLModal extends React.Component { | ||
constructor (props) { | ||
super(props) | ||
|
||
this.state = { | ||
name: '', | ||
showerror: false, | ||
errormessage: '' | ||
} | ||
} | ||
|
||
componentDidMount () { | ||
this.refs.name.focus() | ||
this.refs.name.select() | ||
} | ||
|
||
handleCloseButtonClick (e) { | ||
this.props.close() | ||
} | ||
|
||
handleChange (e) { | ||
this.setState({ | ||
name: this.refs.name.value | ||
}) | ||
} | ||
|
||
handleKeyDown (e) { | ||
if (e.keyCode === 27) { | ||
this.props.close() | ||
} | ||
} | ||
|
||
handleInputKeyDown (e) { | ||
switch (e.keyCode) { | ||
case 13: | ||
this.confirm() | ||
} | ||
} | ||
|
||
handleConfirmButtonClick (e) { | ||
this.confirm() | ||
} | ||
|
||
showError (message) { | ||
this.setState({ | ||
showerror: true, | ||
errormessage: message | ||
}) | ||
} | ||
|
||
hideError () { | ||
this.setState({ | ||
showerror: false, | ||
errormessage: '' | ||
}) | ||
} | ||
|
||
confirm () { | ||
this.hideError() | ||
const { storage, folder, dispatch, location } = this.props | ||
|
||
dataApi.createNoteFromUrl(this.state.name, storage, folder, dispatch, location).then((result) => { | ||
this.props.close() | ||
}).catch((result) => { | ||
this.showError(result.error) | ||
}) | ||
} | ||
|
||
render () { | ||
return ( | ||
<div styleName='root' | ||
tabIndex='-1' | ||
onKeyDown={(e) => this.handleKeyDown(e)} | ||
> | ||
<div styleName='header'> | ||
<div styleName='title'>{i18n.__('Import Markdown From URL')}</div> | ||
</div> | ||
<ModalEscButton handleEscButtonClick={(e) => this.handleCloseButtonClick(e)} /> | ||
<div styleName='control'> | ||
<div styleName='control-folder'> | ||
<div styleName='control-folder-label'>{i18n.__('Insert URL Here')}</div> | ||
<input styleName='control-folder-input' | ||
ref='name' | ||
value={this.state.name} | ||
onChange={(e) => this.handleChange(e)} | ||
onKeyDown={(e) => this.handleInputKeyDown(e)} | ||
/> | ||
</div> | ||
<button styleName='control-confirmButton' | ||
onClick={(e) => this.handleConfirmButtonClick(e)} | ||
> | ||
{i18n.__('Import')} | ||
</button> | ||
<div className='error' styleName='error'>{this.state.errormessage}</div> | ||
</div> | ||
</div> | ||
) | ||
} | ||
} | ||
|
||
CreateMarkdownFromURLModal.propTypes = { | ||
storage: PropTypes.string, | ||
folder: PropTypes.string, | ||
dispatch: PropTypes.func, | ||
location: PropTypes.shape({ | ||
pathname: PropTypes.string | ||
}) | ||
} | ||
|
||
export default CSSModules(CreateMarkdownFromURLModal, styles) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@AWolf81 Instead of doing this, can we use the turndown remove rule?
https://github.com/domchristie/turndown#removefilter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ZeroX-DG good idea. I'll change this in one week as I'm without a computer at the moment.
remove
filter should simplify this.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ZeroX-DG OK, I've changed the filtering. I've created a basic Codesandbox example to check that the
script
tag is not available in Bootstnote.https://4bckd.csb.app/
If you open the URL in the browser it will execute an
alert
and in BoostNote there is noscript
tag available.Is there more I can test?
The
html
constant can be removed as Turndown is handling the conversion string to md - it was just there for the script filtering.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it looks good now. Thank you for your contribution!