Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
E
elp
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Service Desk
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Incidents
Environments
Packages & Registries
Packages & Registries
Package Registry
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
崔为之
elp
Commits
d2952806
Commit
d2952806
authored
Nov 20, 2023
by
崔为之
💪🏽
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Update project
parent
ca65d190
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
12 additions
and
12 deletions
+12
-12
application/dao/log.py
application/dao/log.py
+3
-3
application/services/log.py
application/services/log.py
+9
-9
No files found.
application/dao/log.py
View file @
d2952806
...
@@ -99,7 +99,7 @@ class LogDao:
...
@@ -99,7 +99,7 @@ class LogDao:
_
=
Log
.
batch_save
(
result_generator
)
_
=
Log
.
batch_save
(
result_generator
)
@
classmethod
@
classmethod
def
parse
(
cls
,
start
:
str
,
end
:
str
,
index
:
str
,
sql
:
str
,
cfg
:
dict
)
->
int
:
def
parse
(
cls
,
start
:
str
,
end
:
str
,
index
:
str
,
sql
:
str
,
options
:
dict
)
->
int
:
"""
"""
Parse logs from Elasticsearch and save them to MySQL.
Parse logs from Elasticsearch and save them to MySQL.
...
@@ -108,7 +108,7 @@ class LogDao:
...
@@ -108,7 +108,7 @@ class LogDao:
end (str): The end date for the logs.
end (str): The end date for the logs.
index (str): The Elasticsearch index to get logs from.
index (str): The Elasticsearch index to get logs from.
sql (str): The SQL query to get the intent mapping from MySQL.
sql (str): The SQL query to get the intent mapping from MySQL.
cfg
(dict): The configuration for MySQL.
options
(dict): The configuration for MySQL.
Returns:
Returns:
int: The total number of logs parsed.
int: The total number of logs parsed.
...
@@ -134,7 +134,7 @@ class LogDao:
...
@@ -134,7 +134,7 @@ class LogDao:
logger
.
debug
(
'The data is inserting ...'
)
logger
.
debug
(
'The data is inserting ...'
)
# Get the intent mapping from MySQL.
# Get the intent mapping from MySQL.
mapping_list
=
cls
.
get_intent_from_mysql
(
sql
,
cfg
)
mapping_list
=
cls
.
get_intent_from_mysql
(
sql
,
options
)
# Process and save the metadata.
# Process and save the metadata.
cls
.
process_and_save_data
(
mdata
,
mapping_list
)
cls
.
process_and_save_data
(
mdata
,
mapping_list
)
...
...
application/services/log.py
View file @
d2952806
...
@@ -46,19 +46,19 @@ class LogService:
...
@@ -46,19 +46,19 @@ class LogService:
index
=
current_app
.
config
.
Elasticsearch
.
Index
index
=
current_app
.
config
.
Elasticsearch
.
Index
# Retrieve the MySQL configuration from the current application's configuration.
# Retrieve the MySQL configuration from the current application's configuration.
mysql_confi
g
=
current_app
.
config
.
ExtraDB
cf
g
=
current_app
.
config
.
ExtraDB
# Retrieve the SQL to be executed from the MySQL configuration.
# Retrieve the SQL to be executed from the MySQL configuration.
sql
=
mysql_confi
g
.
Sql
sql
=
cf
g
.
Sql
# Construct a configuration dictionary for the MySQL database.
# Construct a configuration dictionary for the MySQL database.
cfg
=
{
options
=
{
'host'
:
mysql_confi
g
.
Host
,
'host'
:
cf
g
.
Host
,
'user'
:
mysql_confi
g
.
User
,
'user'
:
cf
g
.
User
,
'password'
:
mysql_confi
g
.
Password
,
'password'
:
cf
g
.
Password
,
'db'
:
mysql_confi
g
.
DB
,
'db'
:
cf
g
.
DB
,
'port'
:
mysql_confi
g
.
Port
,
'port'
:
cf
g
.
Port
,
}
}
# Parse the logs using the LogDao class and return the result.
# Parse the logs using the LogDao class and return the result.
return
LogDao
.
parse
(
start_date
,
end_date
,
index
,
sql
,
cfg
)
return
LogDao
.
parse
(
start_date
,
end_date
,
index
,
sql
,
options
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment